How Palantir’s AI Bet is Revolutionizing Defense and Beyond, with CTO Shyam Sankar

0:05Palantir has built a data-driven application platform over the past two decades that is used by governments, militaries, and some of the world's largest companies for analytics and operational decisions. Palantir recently announced a new platform, AIP, as part of a push to invest in AI. This week on the podcast, Sarah and I talked to Palantir's CTO, Sham Sankar, who's the company's first business hire and has led the company for nearly two decades, previously as a COO. Sham, welcome to No Priors. Thanks for having me. Great to be here, Sarah. So I think you have a very unique background. I believe you grew up in Nigeria and then moved to the United States.

0:40You got interested in computers reasonably early. It'd be great to just hear your personal story and your background. Yeah, I spent the first three years of my life really in Nigeria. My father had built the first pharmaceutical manufacturing facility on the continent. Until then, all the drugs were really imported. And we fled Nigeria during some violence and really resettled in the U.S. as kind of like refugees. So a great deal of gratitude, understanding the counterfactual reality of like, you know, how the world could have ended up there. And so I grew up in Florida. I think relevant to the current age, I grew up in a time where when the space shuttles would launch, we would all file out into the recess courtyard to actually just watch it, you know, and that seemed quite normal.

1:22And it also seemed really normal that on Saturday morning at like 6am, you'd be woken up to double sonic booms every now and then. And so I'm eagerly awaiting the return of the space age that commercial space and new space have been bringing back to us here. But I made my way out to Silicon Valley in 2003 and started getting involved with startups. The first company I worked at was Zoom with an X that was founded by Kevin Hartz and it was international money transfer company. And then after three years at Zoom, I started at Palantir as the 13th employee, really the first person on the business side and have had the most fantastic ride ever since, but never been more excited about what we're doing than what we're doing right now, with AIP and the opportunities that are in front of us.

2:07How did you originally find Palantir? Because, you know, it's a very secretive company very early on. It was sort of a very small community in technology and Silicon Valley had actually heard of it. So I was just sort of curious, like how you connected with the company and got involved. One of my friends, so Peter was also a seed fund investor in Zoom. So I got some exposure there. And one of my friends was actually a freshman year roommates at Stanford with Joe Lonsdale. And so I had heard about this company that was very small at the time. And it really talked to my heartstrings. My uncle was a victim of terrorism.

2:39I was in New York during 9-11. And it just like, this was going to be, I would rather fail on working on a problem this important than do anything else. Yeah. And then you joined, as you mentioned, as employee number 13, the first business hire, et cetera. And you've had a variety of roles over time. Could you explain a little bit how your role has changed over the lifetime of the company? I think the kind of common thread through it all as just doing what you need to do, which I know sounds like a banality. But really, when it first started, I wrote our first kind of candidate management system.

3:10What are you doing as a business hire before you have really a product here? And I was an aggressive QA tester, you could say. But the real initial contribution was what we call forward-deployed engineering. It comes from an insight that Alex kind of had around. Like, well, you know, he muses, why are French restaurants so good? Well, maybe one theory is that the wait staff It's actually part of the kitchen staff there. You know, it's not that they have like deep context and understanding of the food. And so the Ford deployed engineering idea was that the people who are going to be interacting with customers in the field were going to be computer scientists.

3:45You could actually understand what does the product do today? What does it need to do today? How is it under what conditions is it going to work? And how do you kind of create this hybrid role that's product management, customer success and engineering all in one? And that's really the team that I first built up. And then as we went from Gotham and Foundry and now AIP, there's a lot to do there. Whether it's interacting more often with the customers, thinking about the technology, or really thinking about how are you going to get to value. And I think a lot of what forward deployed engineering really gets you to think about is how do you do these things backwards?

4:20Instead of really going forward from the technology, how do you work backwards from the problems that your customers actually experience? And use that to create an accountability function. Like, is what I'm doing mattering? Did I make the life of this customer better today? Can I do better tomorrow? And using that absolute standard to judge yourself rather than saying, you know, did my software work according to spec? Like, who cares about the spec? Who cares about what my ambition was yesterday? What's my ambition today? And why shouldn't it be bigger tomorrow? That framework is super practical.

4:49Was it the customers that you were working with early that drove the need for this? Were you trying to solve a particular problem? One, the customers working within government is just motivational, right? Like you kind of view it and you're thinking about it. Like, how do I walk as many miles in their shoes as possible? Like, how could I possibly just think my job is done because I checked the box here? It's like the job is defined by how are they doing and what more could I be doing for them? Then I think there's a more kind of cynical component of this, which is like, okay, well, are you as a company going to succeed in this sort of environment, particularly with early government customers, if you don't have that sort of mentality?

5:25Because if you think about the vertical stack, you need to deliver your outcome. you're dependent on so many things going right. So if you just want to build the software at this component of the stack, if anything down here isn't working, and certainly at the time we were doing this, there really wasn't AWS yet, but you couldn't depend on any of that stuff working. And so your visualization of what you would need to own so that you succeeded needed to be quite ambitious. So before AIP, the company had three key platforms, Gotham, Foundry, and Apollo. Could you tell us about what these different things do for the sake of our audience, since they're less familiar necessarily with the company?

6:01And then how does AIP fit into this? Yeah, Gotham is our flagship government product. It's really focused on intelligence and defense customers, and it helps them integrate and model their data to really drive decisions in the context of their enterprise. So in the defense community, that'd be thinking about in terms of the kill chain, how do I go from targets to effects on those targets? And intelligence, it's often kind of a different sort of structure, but how do I track and gain context and understanding of the things in the world that I need, I have a responsibility to understand about. You can think of Gotham as kind of conceptually at the highest part of the stack.

6:34Much of Gotham then depends on Foundry. Foundry is our general purpose data integration platform. It allows you to deal with structured and unstructured data, to transform that data, to really treat data like code, and then drive that through to an ontology, a semantic layer that models not only the nouns of your enterprise, like the concepts that you think about, but the the verbs, the actions. And so I think, you know, the buzzword for this often digital twin, and that can mean a lot of things to a lot of people, but, you know, how do I have some sort of conceptual understanding and model of what we do as a business and use that to actually affect decisions and then drive that all the way through to the application and decision-making layer.

7:12So not dashboards that give me visibility, but really pixels that I can make changes, right? So if I want to allocate inventory, I need a platform that's going to allow me to write back to SAP or read and write from my transactional systems and orchestrate my enterprise. And Foundry, I think, really gives its customers the ability to kind of squint and model their enterprise as a chained series of decisions or kind of like a decision web, and then giving them the modeling and simulation ability to understand and ask counterfactual questions. What happens if I do this? And this is the same platform that was used to build the COVID vaccine response distribution in the US and the UK.

7:49same platform that commercial companies were using to manage the supply chain crises, when suddenly steady state kind of equilibrium wasn't really there. And being able to model the counterfactuals became really, really crucial. It's kind of interesting that you mentioned those various customers, for example, on the COVID vaccine distribution side or things like that, because the perception of the company is that very early on, a lot of the earliest customers were intelligence and defense. And then it kind of broadened from there. Is that a correct assessment? And was that intentional at the time?

8:17Or was it just you found that there's a pocket of customers that really cared about your product and, you know, were a good fit for what you were doing initially? Well, yeah, we founded the company to work with intelligence and defense organizations. And really, I think we expanded almost reluctantly. You know, I think it was like 2010 or 2011 where we started working with our first commercial customer. But really what we realized was that it took something as sexy as James Bond to motivate engineers to work on a problem as boring as data integration. But this sort of, we had our own ideas of what would be valuable in these spaces, and we built software for it.

8:50But all of those ideas kind of presupposed that the data was integrated. And, you know, I think the kind of popular view is like, this is a boring and solved problem. But I think it might be kind of a boring and highly unsolved problem that people are kind of like duct taping together everywhere they go. And so by productizing a solution to that, we kind of expanded our market and what we could, sorts of problems in the world that we could go after. Apollo is quite an interesting platform as well. So like we really originally built Apollo for ourselves. If you think about our customers, we're deploying in air-gapped environments.

9:23So how do you deploy modern software when you can't see ICD to the target? We had to build this entire infrastructure that allowed us, because our software, it's modern software. We have 550 microservices. We're releasing multiple times a day for each one of these services. They need to be able to upgrade independently, but also our environments are complicated. Like, hey, the submarine only wants to upgrade on these windows or these environments are not connected to the internet. So how is that going to happen? We had to build kind of what we think of as kind of a successor to CICD, which is like autonomous software delivery and deployment.

9:56So Apollo allows you to think about your software and the environments you're deploying in separately, model the dependencies, and kind of hand that to Apollo to manage and orchestrate the upgrade. It will understand what's your blue-green upgrade pattern? How do you think about health checks? How do I roll forward? How do I roll backward? How is that integrated with my understanding of vulnerabilities and CVEs? When do I need to recall software or block software? And that software has started to get a lot more traction as people are dealing more and more with complicated environments, not only air-gapped customer environments in defense and intelligence, but if you go to Europe where people have a strong push towards sovereign SaaS or a lot of people want you to deploy inside of their VPC as a SaaS company, how are you going to manage now having a thousand customer environments to manage?

10:41And Apollo just makes that really easy. And then how does AIP tie into this? And can you tell us more about AIP and that initiative? Yeah, AIP allows us to, it's really a course set of technologies that allow you to bring LLM-powered experiences to your private network on your private data to drive the decision-making. Everything from how do I integrate this data, but I think much more interestingly, how do I build these AI-enabled applications? It's like an application forge. And a core part of the proposition here is that these LLMs, they need tools. Certainly, there's something quite magical about this kind of non-algorithmic compute.

11:19It's neither human thought nor kind of traditional computer science. And they're very good at what they're good at. And they're also quite bad at what they're bad at. And so like getting this right is really about providing, sometimes people call it plugins, but I think tool might be a more appropriate word. But how do I give my customers not just a tool bench, but really a tool factory to go make the tools they need to get the most out of LLMs? Like an LLM is not going to know anything about orbital simulation or weapon hearing or predicting forward inventory 30 days from now. It's certainly not going to do that well.

11:50But with the right tool, it's going to do that quite excellently. And it's going to give you a lot of leverage on the workflows that you already have. And I think in some sense, much of the foundational work that we've done with Foundry has enabled people to run really quickly with AIP, build co-pilots that deploy into their existing decision-making surface area. How do I allocate inventory? How do I adjudicate auto claims and then get that efficiency in weeks? At what point did you decide you wanted to make a big investment in LLMs and what did the company do first? I think it was really around Q4, the last part of the last year there, where it just felt like, obviously, the LLMs are exciting.

12:37But what was more exciting to us is it felt like the LLMs were just waiting for something like ontology. It's like to really get the value out of the LLM, the way that we had modeled the world. It's almost like accidentally we had spent the last 20 years really thinking hard about dynamic ontologies, how you model them, why they're valuable to humans. And you can kind of think about the ontology as having this semantic layer that gives you an incredible amount of compression that you're putting into the context window and allows you to build LLM-backed functions in very reliable ways. And I think part of this is just like recognizing the LLMs.

13:10It's more like statistics and calculus. And I think this is one of the impedance mismatches for a lot of engineers who are working on them. They're kind of like they model them a little bit like calculus, a lot like calculus. and then when it works, it works magically. When it doesn't work, it falls off a cliff. So how are you actually going to get this to work when you kind of have this like stochastic genie now? I think you're going to need a kind of a whole tool chain around that that kind of presupposes it's a stochastic genie. And I think the ontology is one of these things that massively grounds your LLM in your reality, in your business context, and allows you to manage that without having to change the model itself.

13:47What are some of those components that you think are the tool chain that you need to sort of model the stochastic genie. And I love that phrase, by the way. I think it's a really good way to put it. So you're probably going to need everything you kind of need with the dev tool chain, but you're going to have to adjust it for the fact that it's stochastic. So you even see it like people call it eval and not unit tests, but you're going to need like, how many unit tests do you need? If you're going to write an LLM-backed function and it's a stochastic genie, how many times does it need to execute before you have confidence that it's going to do what you want?

14:15And then so then you can think about that. That's like day zero. Okay, so I build this thing. How do I think about it? But what sort of telemetry and production log data do I need? And how often am I going to be looking at those traces? And it's like, I might even be writing unit tests against my traces. I guess you could call that like a health check, right? And like, there's going to be a lot more emphasis that you're going to need there as an engineer, as you think about using this. And then there's going to have to be some calibration on the use case. The best use case is going to be ones where when the LLM gets it right, there's massive upside.

14:46And when it doesn't, it's a no-op, right? And so picking those ones, I think, are going to be quite important as you build and tune the specific applications of these. Going back to this idea of an ontology, I feel like I suddenly understand this much better in that there are a lot of companies right now trying to figure out how to take all of their messy, less than perfectly integrated and largely unstructured data and create some sort of intermediate representation that the models can handle well. And if you have something like an existing ontology of your business, then leveraging it with LLMs does feel like a really natural, magical fit.

15:26That's exciting. Maybe to make it a little bit more real, you could walk us through an example of like an AI tool using this tool chain ontology that you're excited about that one of your customers is building with you? Yeah, sure. Sure. I'll just pick an example that was something we worked on recently in Hawaii, which is how do I do automated COA generation, courses of action generation from an operational plan? You know, so the Department of Defense has these O plans, they call them, that are kind of like, that's the other thing. You have these industries where there's just a tremendous amount of doctrine, whether that's pharmaceuticals or defense, where there's so much knowledge and how we want to do things that's been written down.

16:05So you have this O plan that describes the phases of a potential conflict or the key risks and assumptions. And so you might want to do something like a noncombatant evacuation operation. So if conflict happens here, how will we get all the civilians out of a city? Okay, well, we've thought about that and we've written that down. It's in this document. And so how do I then just say, like, build me a course of action to drive this evacuation? Well, the plan has specified the resources that you're probably going to need, the types of resources, the phases, the timing of it, the risks and assumptions you need to worry about.

16:42So then how do I take those words and then hydrate the application state that people use to manage the common operating picture? And that's a big part of what we're really thinking about, which is, you know, I kind of think of like chat is a massively limiting interface. You know, at the limit, prompts are for developers. Now, I think that's, it's really hard. It prompts leak over to users and users sometimes want to chat. And lots of people start with that because of the popularity of chat GBT. But really what I want to do in this context is they're entering a question like generate a course of action.

17:11Allow that is this evacuation operation. And what I'm getting back is not words. I'm actually getting a map with resourcing, a resource matrix and the requisition of the necessary resources. And, you know, I can hit a button and say yes. And that comes to life. And so those are the sorts of experience that we're starting to build with customers now. And on the commercial end, it's really co-pilots to help people. I was just looking at a demo this morning from my team on helping a major auto manufacturer adjudicate quality and claims. So how do I manage the cost of quality, cost of warranty on the production line and post-production when these cars are in service?

17:45Well, I need to be able to cluster these claims and more efficiently understand what supplier subcomponents wear in the supply chain and how do I remediate these issues? How do I drive down my cost of warranty and recall? So building co-pilots that are kind of looking at the text of the claims, understanding the components, helping them identify early indicators and signs of the kind of conditions under which these parts need to be recalled and managed. And there it's really about human agent teaming. I was listening to the, I think you guys hosted like an AIP day, like a set of like demos and presentations recently.

18:20And one of the points of view you take is that the models themselves are increasingly commoditized and certainly more broadly available and available in open source. How do you think about the value that Palantir is building for its customers? Well, I think it's how do you how do you actually use the models to drive these experiences? So there's kind of two ends of that. One is the existing experiences that people have today. So how do I build a co-pilot that's going to help me adjudicate auto claims or help me understand my production process? On the other end of that, it's like, how do I develop trust in the underlying models?

18:54You know, if we go back to the stochastic genie here, maybe we should actually think about these as like slightly deranged mad geniuses. And then, you know, are you going to only ask one of one of these experts to help you solve your problem? Like, how do you think about the configuration of mad geniuses that you'd actually really want to have? I want an ensemble of mad geniuses. I'll feel better about that. Yeah, I think that's probably the correct direction. And I think one that model companies will have a hard time with, right? Because I think they need to have kind of one model to rule them all or directionally, that's where it needs to go.

19:27But if you're an enterprise and you're thinking about, I have high consequence decisions that I'm trying to drive here, how do I do that? And I think certainly if you're living in a chat world where the output's chat, that's valuable. And you want to think about like, where do people agree and disagree? But if you start thinking about actually the output of the model is a DSL or JSON, that becomes even easier then. Now, you can actually parse these things in very structured ways to understand not only the consensus view, but maybe divergent views. And then you're kind of more authentically treating this as statistics and not calculus.

19:59And then that forces you to flow that through to the UI and how you're designing this for actual human users to interact with it in a way that's thoughtfully statistical. So you've said that you don't think that, you know, chat is the be all end all. And it's quite limiting, especially when you think about like these complex workflows, the outputs that are most efficient for your end users. What are you imagining in the near term that turns into? Is it assistance overlaid on the existing software that your customers use, whether it's things they've built with you or software they already have?

20:34Is it just automatic workflows and outputs in that software? Help us imagine it a bit. Yeah, I think the ideal visualization of it is something like, I have an application state and I have an intent as a user. Combine those two things to give me a new application state. Now, that can be very hard. I think depending on the workflow, that might be super obvious. I think if you're thinking about this like a co-pilot for GitHub co-pilot, it's more obvious. I have an application state and an intent and you generate something for me. Now, it becomes less obvious when some of these things get a little more complex and you may need a little bit of hinting and a little bit of user prompting.

21:12That's where I think the art is. So that would be one piece of it. Then another piece of it is like, okay, let's just say that's too hard or it doesn't fit the use case properly. It's not just about it being hard. You have the prompt that is then, the return is JSON or DSL that is manipulating the application state. The whole point is not to give me answers, but you change my app. And then that starts changing how you think about interacting with these things. It becomes a new UI layer. You know, kind of the most extreme version of this is like, why have any UI at all? If you have really beautifully done APIs and you have, let's call it data APIs and ontology, The data that's actually going between these APIs is incredibly well modeled.

21:53I think you can actually use the LLM to generate a lot of the experiences that you want as an end user. Some people have been talking about this in the context of everything becomes this sort of programmatic, agent-driven world, where five years from now or N years from now, you just have agents that represent you as a user with a specific task, interacting with other agents or APIs. To your point, you really minimize the UI dramatically. Do you think that's the most likely future, or how do you sort of think about where all this stuff is heading? from a UI perspective. It's hard to see so far in the future on this, but what I think it definitely does, when I think about the integration layer, like when I, like you look at like the Gorilla's paper and can you can teach, you know, can you fine tune an LLM to basically tell you what API to call with what parameters?

22:35Like, yes, it turns out. And so, okay, so if that's true, what does system integration look like in the future? That's going to be quite different. So then I think it allows you to create more single panes of glass that are actually truly integrated, which is incredibly hard right now. And I think there's some subtle and interesting benefits. He's like Like one of the consequences of a hack we had a number of months ago was that I had an engineer who could build a feature in a couple hours that we had previously scoped. It was on the roadmap. It was a feature that was going to take like two months and two people.

23:01And it's just simply because the amount of UI that was involved was so intensive. You just replace the UI with language, the whole thing changes. So like that's one way of thinking about, okay, well, what sort of UI are you not building today that you actually don't even have to build today? And that you probably have the tools, the primitives in the backend or the application that you can now surface. So I think that's an interesting place to go there. I don't know about the extreme view of like, look, there's going to be no UI, just build every UI custom every time. But I think the ability for a user to get the last mile to be what they need is going to be really powerful.