956: From Agent Demo to Enterprise Product (with Ease!) feat. Salesforce’s Tyler Carlson

0:00Jon Krohn:You've probably heard or even experienced yourself that building AI agents is easy now, and it's true, for demos. But getting from prototype to production, that's where 90 % of developers fail. Welcome to the Super Data Science Podcast. I'm your host, Jon Krohn. Today's guest is Tyler Carlson, an SVP at Salesforce who is head of product for AppExchange, which you can think of as their App Store. We've partnered with Salesforce today to announce that their Agent Force 360 platform, and reliable agentic applications is now available for building on top of. Hear about this and much more about agentic applications in general in this great episode.

0:43Jon Krohn:Tyler, welcome to the Super Data Science Podcast. Where are you calling in from today? I'm in San Francisco. Hey, John, thanks for having me. Nice classic location for somebody working on agents, and you have quite a senior role in that regard. So you're at Salesforce, huge enterprise, and you are their SVP head of product for something called AppExchange and Ecosystem. Can you parse that out for us a bit? Yeah, so AppExchange is the app store for Salesforce. We launched the AppExchange 20 years ago. It was the first sort of enterprise app store for a SaaS product like this. very famously Mark tells a story about going down to meet with Steve Jobs early on in the early days of Salesforce and one of the things Steve told him he should do is build an app economy around the product Mark ended up procuring the App Store domain and we never ended up using it we focus tested it and came up with the brand AppExchange so Mark tells a great story of going down when Steve launched the iPhone and giving him the App Store brand and domain to launch what would become the iTunes App Store.

2:00So we've been in this business for a very long time, building out an amazing ecosystem of ISVs and companies and developers, building things that extend and make the Salesforce ecosystem great. So that's the second part of my title. I'm also responsible for building out the platform that we offer our ISVs to build on, the APIs, the developer tooling, all of the capabilities that make our platform an amazing destination to build apps, agents, integrations to reach the 150 ,000 plus Salesforce customers.

2:38Jon Krohn:I did not know that story involving Steve Jobs. And so Mark, of course, is still your CEO today, right? Correct. Yes. For our listeners who aren't aware of that famous name. And yeah, so if people are in Salesforce using AppExchange to build apps, are those typically or always going to have some kind of sales related focus? It's a broad spectrum of things. So there's a handful of apps and integrations on the platform that are focused on extending the sort of core technology and apps that Salesforce offers, sales cloud, service cloud, marketing cloud, commerce cloud, things like contract management, pricing tooling, widgets that install into and make the implementation of those products better.

3:28But then there are also wholesale companies that have built an entire verticalized app for a particular industry that leverage the rails of the Salesforce platform to take advantage of everything we build to service the Salesforce apps. We're basically making that platform available to others to build apps on. And there are amazing businesses that have been created over the past 20 years building on the Salesforce platform that are also part of our ecosystem. Really cool.

4:03Jon Krohn:So if we have listeners out there thinking about building an app, whether it's sales related or not, what are the kinds of reasons why they should be coming to the Salesforce AppExchange to do that? So the platform is really the core infrastructure that we build the Salesforce apps on. So it's an enterprise-grade platform that we use to deliver sales, service, marketing, commerce, etc. at massive enterprise scale. And I think we're seeing, especially now, a lot of these entrepreneurs and developers starting up with really great ideas. They have these amazing ideas for applications that they can build.

4:37And it's easier than ever to go from idea to sort of this prototype phase or even a working B1 functional application. and you can assemble a bunch of component parts out of, you know, hyperscaler platforms, go to AWS or Google and build this stuff together. You know, and for some engineering organizations at scale, that's the right decision to make. But a lot of the companies that are coming to us are those that want that availability, security model, you know, foundational platform that is easily extensible and, you know, you can use to build these applications, leveraging the last 25 years of investment that Salesforce has made in the infrastructure that runs our apps.

5:27And we offer that as effectively a platform as a service. So you can build whatever you want on top of that platform and even embed now the full suite of products that we offer to our customers, effectively white-labeled under the hood. So you get this immense head start, basically taking advantage of all of the last 25 years of our R &D when you're building an app on the platform.

5:53Jon Krohn:Right. So it sounds like reliability, security, these kinds of concerns are kind of taken care of out of the box. Does it support people writing code, clicking and pointing, both? It's a combination of all of the above. So we have an amazing low-code platform. if that's the style of app that you want to build. You can define your objects and data model through a WYSIWYG schema builder. You can write workflow through a workflow builder we have called Flow that allows you to sort of lay it all out in an orchestrated fashion and do very sophisticated sort of deterministic logic and workflow through that.

6:31You can write UI through drag and drop customization where we ship our own UI framework that you can leverage. So you can build a fully functional application with low code. But all of that capability we offer in a pro code fashion as well. So we have coding languages you can write on the platform. We support Apex, which is our sort of Java-like runtime for writing code. but you can also write now React and other frameworks on top of the Salesforce platform to build your application. So it's a mix of both. And so customers are using all of that when they build an app on the platform.

7:18Jon Krohn:Really cool. And the reason why this podcast is partnering with Salesforce today is not just to tell our listeners about AppExchange, but to tell them about something called Agent Force 360. Tyler, what is that? So Agent Force 360 is our platform really taken to the next level for building all kinds of workflows for the agentic enterprise, what we call it. And the main announcement that we made recently was not just that you could do this historical building apps on the platform that I just talked about, you can actually leverage the full power of Agent Force, which is our agent development platform to build applications for your customers.

8:04So you get all of the investment we've made over the last year in a sophisticated multi-LLM planner service for reasoning and logic, a new scripting language that we've created for deterministic control over the way that agents interact with and work on things, the full contextual layer of Salesforce data model and metadata model, which gives that agent sort of business context and understanding, and the security and permissions model and layer that we offer. So it's really a fully complete platform for building agentic or agentic enabled applications for the enterprise. And you get a whole bunch of the capabilities that we offer to our customers that are using it for building their own agents as part of the offering that you would then ship to your customers.

9:00So you get that core infrastructure, you get the security model, you get the observability and monitoring layer. You get all these things that are actually the hard part about getting from that prototype idea to the last mile of a production-grade, enterprise-ready, agentic application that you can ship to your customers and iterate and develop on over time.

9:25Jon Krohn:Right. So for example, in 2025, we've heard a lot about model context protocol, MCP. But we've also heard that it isn't enterprise-grade, that it doesn't have the kind of security that we need. So it sounds like the way that you've packaged up Agent Force 360, that it has these kinds of enterprise security and reliability concerns taken care of, just like AppExchange does. That's right. So you have this core platform, all of the work that we're doing to solve these really complex and hard problems of how do you give the agent the right access to the right data? Is it operating in a user context or sort of a broader enterprise context?

10:01What context should be shared? What shouldn't be shared based on different types of workflows? And then also all of that, the deterministic logic. I think one of the things that I think agent developers are learning as they're going through this process, it's really easy to build an agent demo. It's a lot harder to actually make that thing work in an enterprise context in the right repeatable way that is going to be required from your customers. So many of our customers, as we've started this process of building agents on the platform for service or sales use cases, a lot of the logic was being written in natural language.

10:41It's like, always do this, never do this. When this happens, then do this. And these LLMs are really, really good at sort of these more non-deterministic style questions. but sometimes they struggle with that always never, you know, if this, then this type logic. And in an enterprise context, that's not really, you know, something that you want. You don't want the agent to, you know, 90 % of the time do that task and then the other 10 % of the time not. And so, you know, we spent a bunch of time developing what we call agent force script, which is a scripting dialogue that allows you to sort of write that deterministic logic and have the agent sort of understand when there's freedom to sort of reason and execute and when it needs to sort of take the rails that have already been pre-delivered.

11:35And in the enterprise context, what you find is that a lot of that logic that's required to do those tasks is sort of already existing in the applications like sales service, marketing that they've already built or used from Salesforce. So it's a really powerful tool set that we're unlocking for developers to build on.

11:53Jon Krohn:Really cool. It sounds like this is something that allows people to go really quickly from prototype to something like a real world product that they can trust, which I expect a lot of our listeners when they're kind of thinking about, you know, what AI product could I be building or AI feature could I be building, especially if it has agents, probably a lot of the time you're thinking the hardest part is building a good prototype. But is that really what you're seeing? It seems like, you know, this kind of solution that you've built is great for going from once you have that prototype, having something in production.

12:28Yeah, I think it's, we definitely offer tools that make it easy to build prototypes, to build agents. We offer, we have a product called Agent Force Vibes, which allows you to sort of vibe code these types of things on our platform. So we're very much leaning into that direction of like getting from idea to, you know, V1 is so much more compressed now than it has been in the past. But the reality is that that last mile from, you know, V1 or prototype to production grade, you know, ready to be rolled out to thousands or hundreds of thousands of users in an enterprise context is where the rubber hits mile than it is to get to the prototype.

13:12It used to take us a lot longer to get to prototypes. So that like LLMs and what we've done with agentic and vibe coding has compressed that timeline so significantly. But that last mile is still the really hard and challenging part. So this is really, you know, an attempt, but for us to leverage our platform to make that last mile easier for those that want to build on that platform.

13:32Jon Krohn:Yeah, it sounds powerful for sure. I would expect that most of our listeners, when they think about putting their application into production, they would think to go to AWS or GCP or Azure. Why should they instead be considering Salesforce? I think it's all dependent on the specific use case you're trying to deliver. I think in the context of the customers that we have building their apps on our platform, a lot of it is a combination of things. They want the app model and the structure of something like a SaaS application that also has this sort of agentic component with it. So we see these amazing companies that are spinning up, building apps on the platform like a company called Harvest Path that is focused on the meat industry.

14:21And they have built their SaaS application for ranchers and farmers to manage their cattle, all of these different very specific and niche functionality that's all of that is an application with workflow and context. And then they're building this agentic layer on top of it that allows for all of that to become part of a agent workflow. And so if you've already built that app stack and it's running on AWS, yeah, it probably makes sense to think about how you build something on top of what you've already done. But we're seeing a lot of these new emergent sort of vertical companies that are like, I can go so much faster to that both app and agent layer when they build on top of this platform because we're giving them not just the, you know, build an agent, but also build the underlying workflow and app infrastructure behind it in a very fast way.

15:25So that would be my main answer is I think it's not really an apples to apples comparison. If you're trying to build an app where you run the infrastructure sort of all the way down to the server level and you have an engineering team that wants to manage uptime and security and all of that stuff, that's where you would go build on one of the hyperscaler platforms. But if you're a small development team or just a founder who wants to get an idea out to market, you can go a lot faster leveraging a platform like Salesforce.

16:02Jon Krohn:Ready to shape the future of AI collaboration? MongoDB is hosting an in-person hackathon in San Francisco on January 10th, focused on building tools and techniques that let AI agents work together toward complex goals. Teams of up to four will compete for over$30 ,000 in cash prizes, and finalists get invited to showcase live at mongodb.local San Francisco on January 15th. You'll pitch to top names in the industry, an opportunity you won't want to miss. Use the code MDBbuilder to get 50 % off tickets. Space is limited, so apply now in the show notes. We've got a link to the MongoDB agentic orchestration and collaboration hackathon page with more details, including how to apply.

16:45Jon Krohn:Yeah, it sounds really cool. Moving a bit beyond just the Salesforce offering that we're discussing today, kind of more generally to what you're seeing in the marketplace, what are some of the common mistakes that you see when innovators like the founder you just mentioned, maybe a founder with a small team, they want to get from prototype to production quickly? What are the common mistakes that they make when trying to commercialize, productionize AI? I think a lot of people are too focused right now on what I would kind of call a distraction, which is sort of the LLMs, their performance. They want to use the best in class.

17:18And I think the reality is, especially in these sort of like focused markets of things, it's like really focus on what the job you're trying to solve for your customers is. And LLMs are an amazing tool, but they're not sort of a panacea. They're not going to solve every problem. You run into a lot of those last mile problems on deterministic versus non-deterministic logic, hallucinations, other things. And what we're finding is the reality with building a commercialized AI application, it's no different than building any other product that you've ever had to build before. You really have to think about what are the requirements?

17:57What are the jobs to be done? How am I solving something unique for that market or that customer base? And you don't necessarily have to use an LLM for every piece of that journey. And a lot of the tools that we've had at our disposal, code for deterministic logic, database structure for application objects and all of that, standard security management, permissions, all of that stuff is also part of the picture. um and so i think too often i see people trying to just sort of like look at an ai problem being like i want i want into this hype and i want to join the ai train and focus on something really sexy and that idea but there's just so there's still so many problems in little niches across the world that um are right for someone to to just build something to solve um and uh it doesn't have to be quite as maybe big AI problem as they might think it does.

19:06Jon Krohn:Yeah, I love that perspective on trying on the way to build a product. Right now, I am currently experimenting with different product ideas. And my kind of my going theory is to build something that one company is like, this is exactly what I need. Right. Just really get in that niche, start there with some point solution, get it to be secure and reliable relatively quickly like it sounds like it could with salesforce and have them be able to kick it kick around with it if they're actually using it you might be on to something and you can expand from there yeah it's a perfect i mean the the company that i mentioned harvest path um family ranching business and they built this application just to manage their own cattle and to manage their own ranching business and you know figured that this problem was something unique to them that's sort of unique to this industry.

20:02And they could figure out how to take that same concept. And if they could solve the problem for themselves, they could solve it for others. So I think that's a very good way to think about it, which is really like digging in on one very unique problem and figuring out how to solve it better than anyone else is solving it. And if you can do that, you have a business.

20:21Jon Krohn:Do you see AI agents replacing traditional, quote unquote, traditional applications? Or are we going to be in a long phase over the coming years where we are both coexisting together? I think the reality is agents are applications. Like, and thinking about agents in the context of just like you're sort of like an unfettered LLM with access to a bunch of data through MCP. And that's like going to somehow replace the need for deterministic workflow, permissions model, database structures, structured and unstructured data. All of that stuff is deep in context that's required for those agents to do work.

21:02And the reality is, as you look at anyone doing AI agents at scale and at some level of sophistication, it's a multi-LLM orchestrated application with tons of logic built in that determines when to use which LLM for what kind of task, when to use the non-deterministic LLM versus a deterministic task. where is the data stored? What is the context? What's the permission model? I mean, that's an application to me. I think the reality is we're just going to start to see applications look very different than they did before, but I don't buy into this idea of agents versus applications and one's going to replace the other.

21:49I think you're going to see a harmonization and merging of all of that and sort of just what the future applications look like.

21:55Jon Krohn:Yeah, it makes perfect sense to me. So these agentic applications are still just applications with all the same issues as before, but now you have LLMs in some cases being able to handle natural language fuzziness or other kinds of things that would have been magical just a year or two ago. While simultaneously, of course, if it's agentic, there has to be some level of autonomy where it's doing some kinds of tasks on its own. All right, Tyler, this has been a great episode, and our listeners wouldn't know this, but we did this without any retakes, which is pretty cool. Uh, just straight on through the conversation.

22:30Jon Krohn:Uh, before I let you go, do you have a book recommendation for us? Um, so I'm a big, uh, fantasy novel nerd. Um, and I have been a huge fan of Brandon Sanderson for a very long time. Um, I was just recently, uh, reading one of the new entrants to a, you know, this is deep nerdy, fantasy novel stuff of multi-thousand page novels all stacked together. But yeah, I don't have a specific book recommendation, but if you're into sort of sci-fi fantasy and you want to, you know, take on a, maybe a multi-year reading project, Brandon Sanderson would be my recommendation for anyone, anyone listening. He's an amazing author, incredible world building and some really cool premises and context in his books.

23:19It's awesome.

23:20Jon Krohn:Nice. Great recommendation. Thank you. And for our listeners who now have been listening to this episode and love the idea of AppExchange, of AgentForce, how can they get started today? Where should they go? And maybe how should they be following you for your personal thoughts after this episode? I would love for folks to follow me on LinkedIn. We share a lot about what we're doing on the AppExchange and the Salesforce platform there. I think an easy place to get started with building on Salesforce is Trailhead. So we have a website, trailhead.salesforce.com. This is where you can go and get hands-on with the Salesforce platform.

23:59You can get free development environments, do a whole bunch of hands-on enablement and training. There's a bunch of content on there on how to get started building your first agent, building your first agent app. That's a great resource to get started. If you're a deep technical nerd like I am, you probably could just drop into the developer documentation and start reading. That's another great place to get started. I love going and checking out the latest in our developer documentation. So those are two places that I would reference. And if you're not familiar with Salesforce, just go check out salesforce.com and see what we're all about.

24:37And you'll learn more about AgentForce and what we're building for customers there.

24:41Jon Krohn:Nice. We'll have links to salesforce.com. Of course, not a hard URL to find, but also this trailhead.salesforce.com is the beginning of a journey for folks who get to that trailhead. Really cool. Tyler, thank you so much for taking time out of your busy schedule today to chat with us. And hopefully we can catch up with you again in the future and see how the journey is coming along. Awesome. Thank you very much, John. Well, that here was a slick, informative episode with Tyler Carlson in it. Tyler covered how Agent Force 360 provides the infrastructure to take AI agents from prototype to enterprise-grade production, including security, observability, and deterministic control through a scripting language called AgentScript.

25:23Jon Krohn:He also talked about how too many AI developers obsess over which LLM to use rather than focusing on the job their customers need done, and you don't even need an LLM for every piece of functionality anyway. He also talked about how the future isn't agents versus applications. It's a harmonization where sophisticated agents are orchestrated applications with multiple LLMs, permissions models, and both deterministic and non-deterministic logic working together. I hope you enjoyed the conversation. To be sure not to miss any of our exciting upcoming episodes, subscribe to this podcast if you haven't already.