#279 Matthew Carroll: Immuta’s Approach to Secure, Scalable Data Access in the Age of AI

0:00So think of Immuta as a platform that kind of orchestrates the whole provisioning process. We're getting tailwinds around this movement around AI, not because we're magically going to secure AI, but rather because the nexus of what created what we call our first era was this concept of separating policy from platform. They run these very large clinical trials and studies and for primary and secondary analysis. They will use Immuta now as a blinded solution, meaning they blind between the patient, the provider and the researcher. And we act as a blinding system. If that fails, or we can't prove that those blinds were in place, it does not pass through the stages of the FDA cycles.

0:44In business, they say you can have better, cheaper, or faster. But you only get to pick two. What if you could have all three at the same time? That's exactly what Coher, Thomson Reuters, and Specialized Bikes have. since they upgraded to the next generation of the cloud, Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds. How is it faster? OCI's block storage gives you more operations per second.

1:35Cheaper? OCI costs up to 50 % less for compute, 70 % less for storage, and 80 % less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds. This is a cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free. Head to oracle.com slash IonAI. IonAI, all run together, E-Y-E-O-N-A-I. That's oracle.com slash IonAI. Okay, so let's talk about Immuta. Everyone knows that data is the big proprietary piece for most enterprises when approaching generative AI. And I guess data security is increasingly important and complex because the attack surface seems to have spread with all these generative AI apps.

2:56Is that what you guys address? Yeah, I think data security is definitely one aspect of the lens. I think the challenge is there's different roles when it comes to access to data, right? There's definitely a security aspect, which is adversaries trying to steal your data for intellectual property, steal it to use it against you, right? You know, like many other things, but it's equally just there's regulatory risk, right? So especially like clinical trial management, as you can imagine, a pharma is like, how do we do blinded studies in the cloud, right? you know or if you go into a bank like in the cloud we converge all this data about how do we separate buy and sell sides so there's not conflict of interest so there's this kind of governance element of it so if you think of security you've got governance and you've got legal and so the problem is is I think everyone in the world kind of gets security but not everyone in the world really gets the governance and legal aspects of it right and I think as AI specifically generative AI has taken off, what it's done is lowered the barrier of entry for how many people want to get access to data, right?

4:12So that's really the interesting thing. It's not that the data going into the AI is maybe such a risk. It's actually a lot of the generative AI has allowed everyone from someone on a manufacturing floor to the CEO that underneath the covers can run queries and code against data. So you're going from hundreds of highly technical users to tens, if not hundreds of thousands of data consumers that want access to raw data. And so what used to be a security kind of very deterministic rule set for a very finite set of people is now a kind of a gray area that governance and legal kind of have to manage.

4:55So think of Amuta as a platform that kind of orchestrates the whole provisioning process for those consumers. And we're kind of hitting, we're getting tailwinds around this movement around AI, not because we're magically going to secure AI, but rather because there's a lot more people that want it. So we help the scale of making determinations of who can see what, when, where, why, how. Yeah. So it's a permissioning platform. Correct. Yeah. Yeah, that's how we started. We started, you know, we spun out of the U.S. intelligence community. And believe it or not, there's more lawyers than you could ever imagine in the intelligence community.

5:36The government has no shortage of that. And what kind of the nexus of what created our kind of what we call our first era was this concept of separating policy from platform. And so as a couple of intel agencies started investing in real cloud, right? So Amazon Web Services, you can look it up. It was called the Commercial Cloud Solutions, so C2S. They just threw data in S3 buckets. But the problem was it was the same data, but because it was two different, kind of think of it as business units, right? Same data, but each business unit, the way they collected it was different. So therefore, there's different rules as to who can see it, right?

6:17But a data scientist, they don't think like that, right? They're just like, I just want to merge the data together, get some insight, right? And when they went to do that, the whole policy broke down. Because the question you have to ask yourself is not just what is the right policy, it's who makes the determination when policies conflict, who gets to decide which policy wins, or is it a third policy? And it wasn't clear. There wasn't a business process to handle that. So if you think about what Amuta, when we first started, it was this, how do we build an abstraction tier? Rather than writing security logic and code inside a database of what that policy is, like mask this column, remove this, whatever, depending on this user, we built an abstraction layer where a lawyer or a data owner without knowing how to write code can say, basically do that, where they can just click, click, click and build policy on it rather than writing a memo.

7:13and it would enforce that policy as that end user is querying the data. So that's what it gave birth to us. So that's what we call the separation of policy from platform, and we then created, that's what Immuta started as, yeah. Right, okay. Can you give me a use case to illustrate that? I was following you until the click, click, click. Yeah, yeah. So let's get to the click, click, click. So it's really important when you understand like governance teams, I always say, woe is the data governor, especially in this age of AI. They're very capable people, 10 to 20 years experience in their domain.

7:56I'm going to take you through like a real world example in health care. But like these are potentially PhDs, potentially JDs. So they may have both. So they're very high IQ intellectuals that have worked in a domain for a very long time. And they're experts in their field and they understand the data very well. Right. So in life sciences, these might be eight to 12 of these people in a hundred thousand person pharmaceutical company. So there's a real customer, real world example. And they've got to manage clinical trial data. right which is the bread and butter of how they design medications that they want to take to market right and they have to go through a traditional FDA cycle all the approvals and they run these very large clinical trials and studies and for primary and secondary analysis they will use a muta now as a blinded solution meaning they blind between the patient the provider and the researcher.

9:01And we act as a blinding system. If that fails, we can't prove that those blinds were in place. It does not pass through the stages of FDA cycles. That's billions of dollars. That's it's very bad. And the way they managed this before was they would isolate each study as a separate database in like a SAS, SAS Institute. So like a SAS database. And then they would physically move it around. Now they collate it all into like a snowflake or database. So what they do is in Immuta is we can hook into an identity system, pull all the metadata about the users, who they are, what groups they belong to, what's their role, what attributes they have about them.

9:44Like they're an investigator on this study. We also pull all the metadata off the data, right you know what are these column tags how is it sent what's the sensitivity level which study is this part of right um what are the different types of patients in there we merge that together and based on that metadata rather than that governor the governors are like i said they they know the law they know the regulations they know all these things they use that metadata so rather than instructing someone to write code as to how to enforce these blinds, what they do is they go into our GUI and they can say, for these types of users, let's just say a principal investigator on this drug that they're building can see this type of data based on this metadata, right?

10:32And it'll dynamically filter. It'll dynamically mask. It could do things like canonization where we will nullify certain cells. If like, for instance, someone in the trial is seven foot six and is an outlier, right? It'll self-identify outliers and remove what we call indirect identifiers. So because like height is an indirect identifier, it's not important to the study, but you can figure out who that person is because there's only one person that tall. But the idea is they can go and argue and use the metadata about the the identity and the data to create a policy. And that policy could be three different types of policy.

11:14The first is a subscription. So who can even know about the data? The second is a data policy. How do we put masking and redaction and those types of techniques to remove utility out of the data? And then the third is a purpose-based. So are you asking a question of the data by which the reason you're asking is the same reason we collected the data? And so it's a purpose base. And that's important for like the GDPR and data sovereignty, et cetera. So that's what I mean when I say click, click, click. What I mean is there's literally a GUI to where they don't have to write code. They can be a lawyer, they can be a scientist, and they can build the controls without really having to understand any of the data infrastructure or having to write code or be that technical.

12:04That's what I mean by that. And to finish that point, what I would say is that's important because they can make it global, right? Meaning it's not just one study. It's not just one data source. But you can do this consistently across all of your data landscape, which is ultimately how you have to scale in converged architectures. What happens when someone, a user, runs into a roadblock somewhere, doesn't have permission, or certain data is hidden and they believe they have the right to see it and they need it? then can you go to the governor, the data governor who's managing this dashboard and have a discussion with him?

12:57Yeah, that's a great question because that's the pain, right? So as we move forward in our journey as a company, going from 2015 to like 2020, that's the core pain. The problem has always been they a lot of these consumers had to be extremely technical because the only way for them to understand that they they're not getting all the data and there's actually a policy in place. You would have to be technical. You'd have to write code and basically get a denied entry. And then you would have to reverse engineer. Well, who's the data owner? Who's you know? And you have to figure it out. Recently, we've built a data marketplace for that exact use case where what we now do is express the visibility.

13:51So you as a user can see what do I not have access to? And then we can then bring governance and legal into the same platform. and you can have a dialogue between one another and a discussion. And when you do request, it's policy mediation. So there's a policy exception process. And that policy exception process is a negotiation between the data consumer, governance, legal, and security. It's like, how much utility do you need out of this data? Like, do you really need this unmasked? or do you just want it unmasked? And there's a difference. And so prior to Immuta, like all these companies, they would just create a ticket in service now.

14:42And then IT would then make a determination and they make a copy. So there was no negotiation. There was no like, so you're spot on in the sense of the pain is, again, it was consumer to IT because it was all engineering. But now that the consumer isn't an engineer, in all cases, governance and legal need to have a conversation, a non-technical conversation. And so, yeah, we provide a mechanism for them to learn about what they don't have access to and then create that negotiation to be able to ask for more of it. Yeah, I would imagine that your business is growing because data is becoming increasingly complicated in where it lies and how it's connected.

15:37And presumably, Immuta, you can connect all different data sources to the platform. I mean, there's companies have, you know, everything from vector databases to S3 buckets you mentioned. How does all that get piped into the platform? Yes, it's really important. So there's kind of two elements you hit on that I think are architecturally really important. So the first paradigm is this concept of native versus proxy. So modern middleware is changing quite quickly. So the way we used to operate prior to the age of AI, and what I would say is the era of agents, the agentic architectures is where I'm concerned.

16:33And so when you think about high transactional volume, that basically, whether it's a human or machine, there's just lots and lots and lots of queries on data. performance is tantamount, right? It's kind of like, could you imagine sitting at a cashier waiting five minutes or even three minutes for your credit card to process? It would be awkward. So that's kind of the same feeling. Three minutes isn't a long time, but it is in the sense of high transactionality. So to act as modern middleware between cloud data infrastructure or cloud storage and the end consumer, you have to be native. So the proxy design models where you kind of act as a go-between classic SOA architecture where you have web services and so API to API to API, an app connects to you, you connect to the data store, that doesn't work.

17:26It breaks. So you have to natively integrate inside the compute infrastructure. So inside a Snowflake, inside a Databricks, inside a Redshift, inside S3, right? Like whether it's unstructured or structured, you have to be in the compute pattern or you have to create a compute pattern that can break that query out and scale it. Right. So that's the first part of scale in these. I call them agentic architectures because I think the AI is all leading towards these, you know, human human augmented, but agentic, you know, kind of operating environments. So that's like number one is so we, Amuta, you know, in my opinion, in our domain, kind of were, you know, like the first to go about that.

18:13Everyone else was trying to do a virtualization model, whereas with us, we are native inside and we were very fortunate to partner with the Databricks or Snowflake like early days, like 2018. So that's important for anyone listening. Native is the only way it works because it's the baseline of scale. The second issue, though, is what people don't think about, which is what happens when I have lots and lots and lots of data consumers? So it's not just humans, but also machines, potentially millions of identities, human or non-human identities requesting access to data, to data products, to full domains of data, et cetera.

18:59And that's where things also break. And so you have to be able, because the amount of tickets coming in, like the pharma I talked about, right now with just humans, they average 200 ,000 data requests a year. So that's 750 to 800 tickets per data governor per week. Now, let's just say you made that 2 million. So the problem is you have to add AI into this natively integrated permissioning system to be able to handle that scale. So that's how a Muta, when we talk about how we integrate into the data infrastructure, there's two aspects of that. The first aspect is how do you natively integrate into the compute so it can break and it's performing just like you ran a query directly against it.

19:57The second, though, is how do you manage that volume of tickets? And you have to provide direct access at the speed of the query, which means you need AI to automate those decisions. And that's what Amute is doing. Yeah.

20:15How does this apply to sharing across international borders? I mean, there are a lot of companies and organizations that are global. Yeah. Can you talk about the governance of that kind of data sharing? Yeah, absolutely. So there's two types of international data law, if you will. So there's data sovereignty rights. And so people, I think sometimes just assume when you talk about data sovereignty, that's all kind of, you know, one kind of big bucket, just like one use case. It's actually not right. So like the laws in China, for instance, and data exfiltration out of China, the People's Republic of China sees data that is produced there as government controlled.

21:10So data is not going to leave there. You can extract certain metadata out of there, but raw data needs to stay in the sovereignty of China. Same thing in like we see a similar type of legislation in Brazil. the the eu with you know there's a lack of treaty between you know the eu and now u.s um you know a lot of people think of gdpr right but that's really not data sovereignty right it's just that's more about um a series of permissions around like use so why did you collect it why are you processing it and there's ways to utilize uh statistical de-identification so you can anonymize data to then export it, right?

21:51And so the question isn't, the tech isn't the hard part. The hard part is actually applying policies consistently as a global company to where is this person, where did the data originate, and who wants to access it, right? Who are they? And you have to make real-time policy decisions on that. And so that means you need to categorize your data appropriately based off of geo zones. And you need to establish purposes on that data as to certain purposes need to enact different types of policy. Because like a purpose is like in Europe, great. We just need to anonymize that raw data, which we can do in flight.

22:36In the case of China, no, we need to run that locally. That becomes really important in analytics. So if you're adding AI, if you're training a model, let's just say I'm a manufacturer as a customer of ours, and they obviously collect a lot of telemetry off vehicles in China. That model needs to run and be housed, trained, validated and run in China on Chinese infrastructure. Now, in Europe, that's not necessarily the case. We just need to anonymize certain aspects of it. And so that model can be running in, let's just say, California, as well as UK, as well as Germany. Right. So but again, what Immute is doing as part of that is we're taking the metadata.

23:25So the data like the metadata on the data. So where is this data located? Where did it originate from? And we can look at the metadata like the attributes of the user. So the metadata of the user, and we can make a real-time determination based on that policy, what that data needs to look like depending on the use case. And we can change that dynamically and change the policy dynamically. And what's really interesting is, and this is where audit becomes really important, is your audit records need to tell you that it was, you know, the data was run, processed, and queried out of China and stayed in China.

24:00But in Europe, we did this. In America, we did this. In Brazil, we did this. Each of the compute systems, though, don't share audit records. And your seams aren't very good at this. So what Immuta does is we unify that audit across who access what data, when, where, why. And we can express that into a report in our audit logs. So we kind of help the data infrastructure layer, and we help those governance teams simplify not just the policy, like I said, but also the control and that policy, guaranteeing you met the regulation, if that makes sense. Yeah.

24:42And how you talked about scale, and we're talking about international now, what are the limits to scaling? Is there, I mean, if you have multiple jurisdictions that you're operating in and multiple data sources and a huge community of users in different markets. How do you scale that? Is the platform elastic enough that it can just scale infinitely Or is there some bound at which you've got to set up a new account and split? Yeah, no, no, no. That's actually kind of the interesting thing about this. While sovereignty legislation is complicated, the reality is the tech is not the limiting factor, actually.

25:51This is why you have to have a native paradigm. The performance and compute has just gotten so good that the tech side of the house is actually not the limiting barrier. It can scale at cloud scale. The problem always, this is, again, we go back to woe is the data governor. And no one's title is data governor. But there's so few of these. You can't scale them. We have an exponential growth in those who could potentially consume data. And we have a linear potential growth, but even that would be cost and effective. Like when you're talking to someone who needs to have 10 to 20 years of experience in data governance and working within your domain, it takes them at least a couple of years to ramp in that corporate.

26:44Because these are big companies, right? And understand how they work and how you make decisions and the risk tolerance of security and legal. And then the CEO may change out and you get reorgs. Like it takes a long time to ramp in these companies. Like a Roche is going to take you five years to really be good as a data governor. Right. And they're one of the best, in my opinion, at these data mesh architectures and establishing data product managers. And so it's like really difficult to scale those people. And they, unfortunately, are at the center of all this. And you can't make access decisions and you can't make sovereignty decisions and you can't make security decisions without the governor approving it.

27:36And so when you've got 10 of them for every 100 ,000 employees, that's a problem. And so the only way to solve that scale problem is to build AI for them and to have agents that act on their behalf based off of their pattern of how they make decisions. And that's the only way forward here. And is there an agentic layer to your platform? That's what's coming. That's what we're building. But the problem that here's there's no magical solution. There's no horizontal governor. Every company is going to have a different risk tolerance. Right. Like talk. Let's like like talk about iconic brands like what, you know, like ignore the like like customers or not.

28:33But like a Disney is going to be very cautious about like their brand. And they're not going to just accept, I don't know, the risk tolerance of, let's just say, maybe like a company like, you know, a high flying tech company like, I don't know, Facebook. Right. I don't know. But they want to govern their data and they want to regulate themselves their way. So you can't collate all of the access decisions and negotiations of how they got to those decisions and build one horizontal system. So you have to create the mechanism by which each company can kind of create one mind of how their governance team works and makes decisions.

29:30And so we are building agents for governors. But what we are doing first is creating the engine to collect all that metadata, to understand this one collective mind of data governors in these companies. And we're doing that through a policy exception workflow. As I said, someone requests, but they don't have everything they need. So they then have to negotiate. And through that negotiation of literally chat back and forth, like all this natural language, and then we can collect the metadata on how they got to the decision and the metadata on the data and the metadata of the identity who requested it and the metadata of the identity who approved it.

30:13And then did legal have to get involved? Did security have to get involved? We mashed that all up. And what Gen AI is really good at is making sense of that and creating a pattern out of that. And so we need to do, what we are doing is forcing companies rather than creating lots and lots of little like edge case policies, like we always have, we need to actually create global policies that everyone has a problem with and then force those exceptions. Yeah. Because that is the way we create the data that creates the AI to allow these governance teams to scale. Yeah, that's interesting. The platform, how does somebody use it?

30:59Is it a subscription SaaS product? Yeah, yeah. It's, yeah, classic SaaS, like truly SaaS. So we have, we can also fulfill via self-managed on their cloud infrastructure, but classic web application, you log in, SaaS subscription. We're not cheap. This isn't a$15 per month per user kind of platform. But so we're kind of like the Apple, if you will, of, you know, data provisioning and data access management. And it's build is based on usage or? Users. So we have two different types of users, very similar to like the business intelligence construct. So, you know, in Tableau, you have those who can create dashboards and then those who consume from dashboards.

31:52In our world, we have those who can create policy and then those who can consume data via our protected data. Right. Right. So that's kind of how we we view the world. And so we have our governors, our data governors, which is like data stewards and those who can build policy. And we have data consumers, which, you know, some of our customers, hundreds of thousands. Right. And so, yeah. Yeah. Are there areas? I mean, where do you see data governance and data availability and data generation going? I mean, it's just, you know, people talked about big data 20 years ago, but this is another scale and another level of complexity.

32:44What's happening now with generative AI is, are tools or platforms like Commuta keeping up with the complexity or do you think there's going to be a further layer of abstraction, you know, I don't know, maybe at the government level or across regions or something where people are going to want to know where data lies and where it's going and how sensitive it is and all of that? Yeah, I think you're on to something. I mean, I think it's going to take quite some time. I mean, I was at the center of the big data movement. You know, like when I first started thinking about big data, it was just as like InQTel just had invested in Cloudera.