How Do Regulated Enterprises Deploy Private, Secure AI?

0:08Aaron: Good morning, good evening, wherever you are, and welcome back to the Enterprise AI Show. This is your host, Aaron, and super quick intro today, we have Ivan Lee, founder and CEO of Datasaur, to talk about what it takes to build private, secure AI for regulated enterprises. We talk a bit about the buy versus rent debate and dig into control and ownership of AI and how that is critical to the enterprise. And all of that is coming up right after this quick break.

0:44Aaron: What does the outside world already know about your company? Cyber criminals could be seeing leaked credentials, compromised session cookies, exposed infrastructure, and even impersonations of your brand and executives. Nord Layer Intelligence by Nordstellar gives security teams visibility into those threats across the deep and dark web. Data breaches, attack surfaces, and brand impersonation all in one platform. Find out what attackers know before they can use it. Visit nordlayer.com slash intelligence slash enterprise AI and use code enterprise10 for 10 % off your Nordstellar plan.

1:32Aaron: Today's show is sponsored by Nasuni. There's a growing gap in AI right now between what's possible in theory and what successfully works at scale inside an enterprise. The difference comes down to unstructured file data. Many AI initiatives struggle because the file data they depend on is scattered, unstructured, and disconnected from where and how work actually happens. Nasuni changes that. It brings your unstructured file data into a single, secure foundation so AI, both generative and agentic, can access it with the context, governance, and performance it needs in production. Bring AI to where your unstructured data lives.

2:06Aaron: See what it takes to activate your data for AI and request a demo at nasuni.com slash AI.

2:15Aaron: And we're back. And for today's topic, it's actually something Brian and I have talked about on the podcast a number of times now, but enterprise AI, but more specifically secure and sovereign enterprise AI. And for that, we have Ivan Lee. Ivan is the founder and CEO of DataSore. Am I saying that correctly? First of all, Ivan,

2:41Ivan Lee: That's correct. Thanks.

2:43Aaron: Cool. I just want to make sure. And what we're going to do is we're going to jump into a bunch of topics here. But before we do that, why don't you give us, Ivan, a little bit about your background? You've got a really interesting background from you had an early exit under your belt and then went over to Apple and now kind of doing your own thing. But I jokingly refer to everyone that's been doing AI before LLMs. You did AI before it was cool. So give everyone a little bit about the background.

3:10Ivan Lee: Absolutely. My name is Ivan. I'm the founder and CEO here at datasore.ai. My own background is that I studied computer science at Stanford. So fun fact, my very first AI class was with Dr. Andrew Ng, and my first computer vision class was with Dr. Fei Fei Li. And so I got to learn from these global superstars before their stardom. As you mentioned, my first company was a very different space. It was mobile gaming, ended up selling that to Yahoo, where I began working on machine learning for the first time. This is 2013, the last AI wave. And I worked on search and machine learning algorithms at Yahoo, then Apple, before realizing that I really enjoyed the world of NLP, natural language processing, and decided to found this company.

3:58Aaron: Very nice. And tell everyone a little bit about, and we'll talk about enterprise AI here in a second, but I'm just really fascinated about the background here. What was it like doing NLP before NLP really was a big part of generative AI, right? Like what was the fit and what was the use case and why were people doing NLP and even data labeling as well back then?

4:23Ivan Lee: Yeah, I think it was just this pursuit of being able to teach machines about our language and how we operate as a society. And it felt like such a fundamental part of how we communicate. And it was just really fun watching machines learn more about us and be able to communicate. So I worked on products like Siri, and it felt like we were on the cusp of something, but even I could not have foreseen the big chat GPT moment. That's fair.

4:52Aaron: That's fair. All right. So let's jump in here. So what I want to do is start at a high level and then we'll kind of keep going lower with each of these topics here. Let's kind of start with data sovereignty and secure AI. The reason why I want to start here is, in particular, we tend to be a little less timely on the podcast. We don't like to introduce all the news because it tends to really, okay, it might age the podcast a little quickly, especially with the speed of AI news these days. But there has been recently the whole idea of, I'm going to use the term forced training, meaning Fable and some of the others where it's like, okay, there's no opting out.

5:36Aaron: of training the data. And I've seen that as one big aspect for data sovereignty and certainly has made this almost like a board level and C-level issue extremely quickly. But even before that, this is something that was super concerning. And so why don't we start there? Tell everyone, Ivan, when you're talking to customers and they say, I want my own AI and I want to talk to you about it. What's the drivers behind that, first of all?

6:06Ivan Lee: Yeah, there are several drivers. I think for some of our customers, it just begins with a compliance story. Their security, their own internal legal teams are turning off chat.com and disallowing some of these external tools because they need to understand how employees are using them, right? And there may be compliance and regulations in place where you're just not allowed to send certain types of internal data to third-party vendors like OpenAI and Anthropic. For others, it may be a little bit more forward-thinking. They realize that AI needs to be a strategic asset, something that continues developing in value over time.

6:47Ivan Lee: And whenever they're just sending all of these prompts and all of these use cases off to a third-party vendor, they're not really capturing that value internally. And then, of course, there's the age-old just straight up price and value, right? We have people saying, Hey, my, my token bills are too high. Are there cheaper alternatives that are going to have better price performance? And it turns out there've been a lot of alternatives released. Yeah. And that's a really good segue into the

7:15Aaron: next part here, because I really want to, you and I both have talked about this and by the way, too, the way we were introduced, I'm speaking together at a few events here recently, but there's almost like this big, like, what is the first top level decision is architecturally, are you going to buy? Are you going to rent? And I want to talk about that, but then I also want to break it down even more and go, okay, why buy? Why rent? Is it as simple as it's the private cloud and public cloud argument all over again, and it's OPEX versus CAPEX? or is it more to it that in the whole buy versus rent?

7:58Ivan Lee: Absolutely. So I think, first of all, for many years now, the default decision has been to rent because there was no alternative solution. You couldn't build and pre-train one of these on your own. Kind of the open weights models that were available just were not nearly good enough. So it made sense and everybody defaulted to this concept of renting. but the landscape has matured and evolved significantly. And in 2026, there are many alternatives now to simply renting from these third-party vendors. And as the concept implies, whenever you're renting, you're always on borrowed resources from someone else.

8:39Ivan Lee: You're on their roadmap. And so that limits your own capabilities to drive what direction it goes in. Again, here in 2026, there are many different open weights models that you can use as a foundation that may be 90%, 95 % as good. But now you use that as your own internal foundation. You can continue customizing it, developing it, and building out your own strategic capabilities over the next couple of years. And we're seeing more and more stories. Companies like AT &T come out openly and say, you know what? we've moved 40 % of our workloads to open weights models. And their goal is to eventually get to 70%.

9:23Ivan Lee: Now, one thing I want to call out there is the goal isn't 100%. It doesn't have to be, right? It's this mixture. The most advanced clients are starting to realize there's certain workloads that we want to send off and continue renting because we need top tier capabilities. But for many of where you're just summarizing a document, where you're just generating some basic email concepts. You really don't have to be using frontier top dollar options. Yeah.

9:53Aaron: And I want to dig into the model agnostic in a little bit more, but I kind of follow on to that. What's the, I'm sure it's very varied at times, but what is the top level decision? A lot of times does it come down to simply is it more like this is a proprietary data and proprietary workflow kind of thing and i want to keep that i'm gonna i'm gonna buy that in my terminology here and i'm gonna build that out because it is crown jewels keys to the kingdom whatever analogy you want to use and then you rent out or go use cloud services or basically pay by the token or paid by the API for the non-differentiated.

10:37Ivan Lee: Yeah, absolutely. So let's take a step back for a moment here. How is this technology trained to begin with? Open AI, Anthropic, they scraped everything available on the public web. And that's a lot of knowledge. That's why these models are impressive. But it turns out that everything that's on the public web only represents 2 % of human knowledge that we have as a society have created. Where's the remaining 98 %? In long lost books, in your personal Gmail inbox, and yes, the proprietary data that these enterprises are holding close to their chest, right? So my silly example here is if you got the KFC 11 spices, I sure hope some employee isn't like typing that into chat GPT and asking questions with it, because that's how you let go of your secret sauce, in that case, literally.

11:33Ivan Lee: And so as enterprises and organizations are becoming more aware of this, they're realizing, why should I continue taking my hard-earned expertise in how to assess an insurance claim and working with OpenAI FDEs and teaching them how to do this? Why am I not just turning that into my own internal AI that I own and nobody else will be able to ever train from? Remember, ChatGPT and Anthropic need to continue evolving. They need to continue fueling their IPO. And GPT-6, Fable 2.0, will be trained on all these specialized finance workflows, these insurance and legal workflows that they're out there trying to learn about as actively as possible.

12:25Aaron: Thank you for that. First of all, I want to kind of move on to the next. What I wanted to do is, and I want to break down when it comes to like making sovereign AI or secure enterprise AI, how do you see the stack? Because I want to talk about models and agnostic models in a second, but I also want to break down the stack. Like NVIDIA famously has the five layer cake. In your conversations and in you building customized agents and workflows for customers, how do you approach it? What are the layers that you see?

13:02Ivan Lee: So I think the important layers are what infrastructure you're going to deploy on. So we start with, does this need to be on public cloud, on your private cloud, on your AWS or Azure instance? Or does this need to go the distance? And we have some clients who insist on bare metal machines, right? So we start with, where will all of this lie? The next layer will be the model itself. What models are available to you and your team? We have some clients who insist on Western trained models, where we would consider models like Google's Gemma Mistral models or the new Meta Spark models. We have others that are more open to which we would be able to consider a lot more of the Chinese open weights models as well.

13:51Ivan Lee: So once you have the model there, the next is this is where things get a little bit ambiguous, but I'll refer to it holistically as the harness, the context. That is the ability for this model to plug into your internal databases and data sources, the ability to call tools and APIs. So, for example, if it needs to connect to your CRM or if it needs to connect to your EHR, is that available as context for each individual call or agent? So at an oversimplified level, perhaps we see those three layers.

14:25Aaron: Yeah, I like that. So when, first of all, I'll ask the question, do a lot of your customers then want this to be actually agnostic top to bottom? There's always the, we famously talk about it on the show of like, customers don't want to be locked in and every vendor on the planet wants to lock you in. And so is agnostic AI even possible? Or is it like only maybe at the model level at best? And tell me a little bit about how you think about agnostic AI. Yeah, absolutely.

15:07Ivan Lee: So I think it was unfathomable just a couple of years ago when it felt like OpenAI and perhaps Anthropic at the time were way ahead of the pack. But we have to accept that in this day and age, models are largely commoditized. Yes, one may be ahead for a couple of months or so. But by and large, if I share with you a prompt and the resulting answer from any of these models, you and I probably wouldn't be able to tell the difference. Or maybe we're very experts and very much experts in the space and we can tell the nuances. But if we prompt engineered it to speak in proper British format, then we really wouldn't be able to tell the difference anymore.

15:50Ivan Lee: Right. And so that's what I can communicate to my clients. We have to be able to train this in such a way that it can easily swap out the model itself from one model family to another, and the rest of the harness will still work. Because we don't know what's happening with these third party vendors. We don't know when one gets sued by the New York Times and has to retrain their model family and suddenly it's lacking certain kind of knowledge and we have to swap to something else. We don't know when something's going to be deprecated. I had a client recently who built everything on top of GPT-4-0 and it was deprecated four or five months ahead of schedule and they were scrambling to retrain everything.

16:31Ivan Lee: So for these reasons, it can be dangerous to be overly dependent on any particular vendor.

16:39Aaron: Yeah. And this reminds me of, for our listeners out there with cloud backgrounds, this reminds me of the days of Kubernetes when Kubernetes was doing a release like every six weeks, like three to four months. And if you're an enterprise, you're not doing that if you're an enterprise you'll be lucky to do big upgrades twice a year sometimes three times a year sometimes and so i think what you're also telling me is it's a way for enterprises to protect themselves from churn because everything is moving so fast but let me ask you the the flip side to that so is so the industry is going to continue to move fast if not even faster going forward.

17:21Aaron: But we're trying to build an AI stack that almost has hot swappable pieces, if I'm using those terms. And also we want to protect our RP. We don't want to give it out to the vendors as well. So what does that mean when it comes to, okay, I've built something, I've built it, I'm going to use that term hot swappable. Like everything's hot swappable. I have my infrastructure pieces, I can do it inside and out. But there's almost this difference between like the tools and the tool adoption and the infrastructure adoption. So tell me a little bit about how you go about convincing someone to that, because for every piece, there's a time trade-off, there's a cost trade-off, like there's lots of trade-offs here in all of this.

18:11Aaron: Don't panic. This podcast is a work of fiction. From the Global News Network, it's 2029. Breaking news. A hacker calling himself the Purple Yak may have used advanced AI to gain control of a Russian nuclear warhead. He's reportedly offering to launch it at a target of the highest bidder's choosing.

18:30Ivan Lee: get the news from the end of the world follow the clues unravel the mystery as civilization collapses 2029 the war for our world a science fiction news series from the future premiering october 30th follow wherever you get your podcasts

18:50Ivan Lee: yeah so i want to take a moment here and empathize with all the employees out there that are being asked to adopt AI or being forced to do so, right? I've been to conferences where a lawyer will come up to me and say, you know what, I have access to 11 different AI tools and five more POCs. And it's really confusing to know which model or which tool to use at any given point. So I want to acknowledge we're in this kind of like frenzy where everybody's testing out a dozen different tools at any given point. And it gets real confusing to the end user. Right. So I think there is this need, this pressure from the market, from the board, from investors to adopt AI.

19:37Ivan Lee: But the way that we're all approaching it can be a little disorganized. It's whoever we hear from first, a headline that we see in the Wall Street Journal. People are just picking things up and seeing how it works. One of the things that has become a trend again in Silicon Valley over the last couple of years has been this concept of change management, just human change management. How do we make sure that it's clear to the end user how to use each of these tools? When's the appropriate time to use them? And we need to stabilize that as an industry.

20:11Aaron: Yeah. And I'm going to take it one step further. I'm going to make the job of complexity even harder. Let's talk about regulated industries like banks, hospitals, law firms, manufacturing, government, all these different ones. They have different regulatory environments and certainly globally even differently within the same verticals. But they kind of all share this common bar of, okay, you got to be able to trust the AI with sensitive data. That's kind of the core fundamental principle around all of this. What does that actually look like from where you sit? Like, how do you approach navigating these waters, if you will, of regulatory concerns and convincing that CISO that is an AI hater that this is going to be okay?

21:02Ivan Lee: It's a tough question. And again, the CISO's role is not easy these days with new cybersecurity threat fronts opening up on every dimension, regulations constantly changing around the world, having to pay attention to what the EU is requiring versus what the state of California or the state of New York is enacting, right? It can be a lot just to keep up with the regulatory front. One of the big advantages of private and sovereign AI is that it does just sidestep a lot of those issues because a lot of the data and the models themselves now live entirely on your infrastructure, on your environment.

21:41Ivan Lee: you no longer have to vet nearly as many terms of service and ongoing changes to which data is allowed to be sent to a vendor. That makes sense.

21:53Aaron: That makes sense. Let me ask you this, because I feel like agents is its own different beast right now in the industry. There's AI and then there's AI agents. And I assume this, but I'm going to ask this with the customers you talk to, how much of this is create me enterprise AI and how much of this is create me enterprise AI agent workflows and break down the differences of how you consider or scope or what you even think about what's top of mind when somebody goes down one of these paths with agents.

22:29Ivan Lee: I'm really glad you asked Aaron. I think that's actually been one of the most important developments of the past year. A year ago, if you and I had been talking about agents, I would say that was these are overhyped and agents don't actually work because in reality, they didn't. There was a breakthrough around December 2025 with around Claude 4.5, where agents went from an R &D theoretical kind of promise to something that actually started working. There was this up leveling where the reasoning actually worked. Why is that important? It comes back actually to what we were talking about earlier with change management.

23:07Ivan Lee: A lot of these AI tools for the past couple of years have really been opt-in. You had to learn when to open a tool, how to use it, how to prompt it a certain way to get the answer that you wanted. And for many, that was quite confusing. The benefit to agents is that they're the exact opposite. They're opt-out. So with an opt-out technology like this, they get triggered by external events. a customer support ticket being filed, an invoice being sent from a business partner. They get triggered by these events and then they alert the employee what action needs to be taken and when. This flip from opt-in to opt-out makes all the difference.

23:50Ivan Lee: If you look at these market reports, the chatbots, the AI tools of the past were 15 % adoption rate. With these agents, suddenly it's 100 % adoption rate. Right. And so what we're trying to push as many of our customers to now is agentic flows because it makes it the adoption curve a lot easier for the end user.

24:16Aaron: First of all, I love that analogy. And I'm also going to take it one step further, though, of, OK, let's say they do that. Let's say you get that first use case and we get into production. One of the big things I'm seeing is there's three big concerns around agents today. Number one, that trust. And what I mean by trust, because I want to break that down slightly, is predictability. Like it's going to get to the answer. It's going to get the answer in the same way. And it's going to get there in a way that's accountable. So if I ever do need to troubleshoot it, I know what's going on. So that's one area.

24:51Aaron: The second area is security. and the third area is really this idea of fin ops and the finances behind it how many tokens is it going to consume if that's how you're metering your environment and so i feel like there's a lot of folks that go wow this is amazing but then they somebody comes in and puts the on the the enterprise hat and goes what about this and what about this and what about this and so how do you handle the after the fact of like the trust and the security and the roi of these right

25:22Ivan Lee: Right. So good questions. Those are, let me break that down from different angles. Let's start with the fit ops and the ROI. On the one hand, the ROI is abundantly more clear because now you're getting that 100 % adoption rate, right? And that for a lot of our customers has been delightful to see that like everything that was promised, they don't have to worry about the adoption curve. It just immediately starts delivering. There is one big caveat to this. The average chatbot query you and I put in to chat GPT, that takes about 2000 tokens. The standard agentic task does take on average 3 million tokens.

26:07Ivan Lee: That's a lot more tokens. That's three orders of magnitude more tokens. And I'm not here trying to like dupe you into using up more tokens. That is a real trade off. Right. And so that kind of ties back into earlier parts of our conversation where it may be important to consider smaller models that are more token efficient, that will have less cost per token, making the ROI still fully viable. Let me pause there before I move on to security and trust. No, please.

26:36Aaron: That's perfect. Go ahead. Makes perfect sense. I love it.

26:40Ivan Lee: So on the security and trust part, that's something that we handle very carefully as well. We create custom benchmarks for every single customer. And we start with, okay, what does your workload workflow need to accomplish? We work with their subject matter expertise to establish a baseline of 50, 100 questions that kind of set that evaluation level. and we won't stop until the agent is able to deliver 95%, 98 % accuracy, whatever you deem appropriate here, right? And so that is how we establish the trust in the broader system. And we continue running that even after it's in production, even as we continue updating the model and connecting to new data sources, that benchmark always has to be met.

27:33Aaron: yep and the one big thing i go to though this kind of goes back to our original thesis of buy versus rent one thing i've heard i'm a devil's advocate this for a second in the buy model yes you get i'm going to own everything all the way down to my data and my embeddings i'm going to own my models i'm going to own my harness i'm going to own everything but now you have to run it. And you have to have the expertise in more particular to be able to run these things. And so this is where, of course, the rise of FDEs has come into play and folks coming in and I don't have the expertise to build it.

28:16Aaron: So I'm going to have somebody come in. Makes perfect sense. But what happens over the life cycle of all of that? Do you have to educate your customers that, oh, by the way, when you build this, this is what day 122 looks like, or day 500 looks like of owning and operating this. I jokingly refer to it as care and feeding. What is the care and feeding of all of this over time? And the rent scenario, a lot of people just go, look, I don't need to do any of that. So how do you handle that trade-off and how do you advise customers on that?

28:52Ivan Lee: It's a great question. And part of it is this is something that only really gets solved with time, right? Just as companies over the last 30, 40 years have developed IT as a critical function, perhaps more recently, kind of cloud maintenance specialists as a critical function of an organization, so too will it be important to have an AI infrastructure team in the future. Maybe by 2040, this will be a midpoint and everybody knows how to maintain this. The question is, it's a matter of timing. When should I begin to invest, right? When does it feel important? And for many, 2026 is that inflection point.

29:35Ivan Lee: It's this kind of back to the very beginning of our conversation. It's this perfect storm of, I don't want to be fully dependent on a vendor like Fable that can be turned off by the US government at a moment's notice, right? I don't want to be tied to a vendor that is constantly training on my data and using this to promote their own financial interests over my own. Or it could just be a matter of, look, it is now financially viable for me to start building and owning my own AI at this point. Yeah.

30:06Aaron: Now, I actually want to ask you what you're seeing in the field, because one thing I've seen is up until now, it was like, oh, if I want to get quote unquote custom AI or my AI, you could do a certain amount of context engineering or prompt engineering. You could do rag and do rag retrieval. And I felt like it was always the bazooka. The big one was fine tuning your own models kind of thing. I feel like this fourth option has come along now, which is harness and being able to steer it with a harness. Where do the customers you speak to sit in that almost like a spectrum of like easiest to hardest and which ones are, what's your opinions on which ones are the most effective these days?

30:57Ivan Lee: Yeah, of course the answer is it depends on each individual project, but at a high level, I would stacker rank those as we try to go, we have all these tools in our tool belt. And we will approach it from lowest hanging fruit to the most difficult one. So we'll start with things like prompt and system engineering. RAG is probably the next easiest. And then Harness has really become the largest way for us to close that kind of evaluation benchmark gap. And then last but not least, I'm still a big fan of fine tuning, but it is a tool of last resort for us. Because once you do that, you're going to have to continue to fine tune all future versions of the model as well.

31:41Ivan Lee: So that's my loose hierarchy. But of course, we will evaluate each project on a case by case basis.

31:49Aaron: It makes perfect sense. Let me ask you this, because I feel like we've been digging into that. The one other thing I wanted to ask you is how do you handle data? Because obviously, if you're talking to customers about their AI and customization of it, there's going to be data, their data, but there's going to be PII information in there. There's going to be sometimes cases where maybe they don't have enough data and they're doing a certain amount of synthetic generation on top of their data, or maybe there is regulatory concerns of things they can and can't do. How do you approach, I'm looking for a term here, I want to say data scrubbing, but it's almost like a data audit or a data analysis of and advising customers on what they can and can't do, and then where and how they store that data.

32:44Ivan Lee: Great question. So that is something that we take very seriously. Perhaps it's because of my background at Apple, but we are privacy first, right? And so we start with the customer's constraints around where data is allowed to sit, who's allowed to touch it, what form it takes, and we'll accommodate. Remember, just because the model is on your infrastructure, that doesn't necessarily determine where the data sits. So if it's already housed in AWS S3 bucket somewhere, we can still have the model access that. If the data also needs to join on the machine itself, we can accommodate that as well, right?

33:23Ivan Lee: And so we will always work hand in hand with the security and the legal teams to work backwards from, okay, where is the data existing? How can we safely access it? If we need to tune, do we have to use, as you've mentioned, de-identified or synthetic data? Or are we allowed to use the actual data just to tune the model weights themselves?

33:46Aaron: Makes sense. Makes sense. All right. So I think that's a great place first to stop. I think we did a good job at starting at the very top and getting all the way to the very bottom through the episode. So Ivan, thank you very much for that. For anyone out there that is interested in this or wants to reach out to you or just wants to get started in general, what's the best way they could get started, Ivan?

34:07Ivan Lee: We can start with a conversation. Please reach out to me on LinkedIn. My tag is IY Lee. I'm happy to talk about Sovereign AI and just schedule a strategic consultation and see what's the best path for you.

34:22Aaron: Fantastic. I appreciate your time, Ivan. And everyone out there, thank you very much for listening this week. We certainly appreciate your time. On behalf of Brian and myself, thank you, everyone, for listening. We certainly appreciate it. And wherever you get your podcasts, if you don't mind, if you could leave us a review if your outlets allows reviews, we would certainly love that. and we're always interested in feedback as well. Feedback details are in the show notes. So again, thank you everyone for listening and we will talk to everyone next week.

34:49Ivan Lee: Thanks for listening. Check us out at theenterpriseaishow.com for past shows, newsletters, and all things enterprise AI.