AIUC-1: Building trust in AI agents

0:02Daniel Whitenack:Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

0:41Daniel Whitenack:Welcome to another episode of the Practical AI Podcast. I'm Daniel Whitenack. I am CEO at Prediction Guard, and I'm really excited today to have an amazing guest that I'm personally interested in asking a bunch of selfish questions to because I'm so interested in the topic. We have Emil Lawson, who's the standards lead at the Artificial Intelligence Underwriting Company. Welcome. How are you doing? Thanks, Daniel. Thanks for having me. I'm doing great. How are you? I'm doing well. Actually, you know, today in the Midwest, everyone's concerned about tornadoes and talking about hail and some things like insurance and other things.

1:19Daniel Whitenack:So it's a whole other world, of course. But obviously, the AI underwriting company, way more than thinking about insurance, but thinking about standards, certification around AI and agents. I'm wondering how you personally, just to give the audience a little bit about you personally, how did you end up at this intersection of standards, certification, AI, agents? How did that come about?

1:49Emil Lassen:Yeah, so I don't think I had as clear a path as the classic standards lead where you work as a, say, security engineer for 10 years and then learn the technical craft and then come together. My journey has been very entrepreneurial always. I started my first company with actually the CEO of the artificial intelligence company, Runekvest, 10 years ago. There was a nonprofit back then helping students from low-income backgrounds get into top universities. And I think what I took away from that was both the very entrepreneurial journey, but also the desire to move fast on some of the challenges that society is facing.

2:22Emil Lassen:I then moved in and had my first interaction with standards at my second company, a real estate company back in Denmark, where we developed an impact management system that both had to navigate a lot of national legislation, local legislation, EU legislation, voluntary frameworks, investor demands. And so I had my first interaction with one of these quite complex markets of different measurements and targets you wanted to get to. A very technical sector as well, where like building codes, for example, require a lot of thinking through how you do things the right way. So I spent four years building up a real estate company that today is managing about$400 million together with four other co-founders.

3:04Emil Lassen:and took from that this desire to go in and standardize when we know what the right answer is and try to push the sector in that direction. The way I got into the AI space was taking a step back from the real estate company now a few years ago and going to Cambridge, Massachusetts, Harvard University, where I spent two years as a fellow at the Kennedy School, really getting into the emerging tech and geopolitics of all of this as well, and left the Kennedy School with a very clear ambition of just getting under the hood of the pace of AI, the safety and security aspects, and clearly just acknowledging that the technology is going to profoundly change society as we know it today.

3:46Emil Lassen:And in many different ways, I have 10 nephews and nieces, I have five sisters, and seeing a 10-year-old being comfortable using AI the way they are. It's kind of scary. And I don't see yet that we've codified the principles we want to see when it comes to how kids use AI. So that's one direction where, oh, maybe we should develop standards for this. You also can read news every week and see new incidents. So there's clearly a big security angle to this as well. You said that the Midwest is facing tornadoes today. I think being a CISO at a company adopting and deploying AI right now feels a little bit like you're in a hailstorm and it's only a matter of time before you're hit by some of that.

4:27Emil Lassen:And we just keep seeing that. So clearly there was a element to that as well. And then there's the even bigger picture around what this will do to our job markets and so forth. So I left the Kennedy School being very interested in just using the public policy toolbox that I brought from that. With the standards toolbox I brought from my real estate company. And then this desire to actually just work on societal challenges that I think has been with me since I started working. And that became my way to the artificial intelligence underwriting company. I've since then spent all my time building a network of people, a consortium, to help them figure out how we get the right practical and technical insights into the standards we develop as well.

5:06Daniel Whitenack:Yeah, that's awesome. And I know sometimes when people hear things like standard certification, you know, terms like this, maybe some people might have a reaction of like slowing things down, right? But I like how at the very, you know, front and center of what you talk about is how to actually unlock enterprise adoption with certification standards, etc. Could you talk, before we dive into the AI side of this specifically, could you talk about that a little bit in general, like how some of these things work together in actual enterprise settings, standard certification, even insurance, and how those things actually can enable adoption and not just like block things, I guess.

5:55Emil Lassen:Happy to. So I think our story and the inspiration we take dates back to Benjamin Franklin's Philadelphia. Philadelphia was starting to adopt electricity. Electricity was scary back then. Light bulbs did not work out. Homes started burning down. So Benjamin Franklin formed the first fire brigade in Philadelphia. He started codifying building codes so that we basically took what we knew around how to build safer houses, the standards part, and then he developed the first mutual insurance company. So back then, this is the first time we see this flywheel of standards, audits, and insurance go together.

6:31Emil Lassen:By having standards around building code, for example, you knew that houses needed to be placed a little bit further from each other. They needed some of his lightning rods to ensure that when lightning strikes, that they don't catch fire. The fire inspections was the audit part that actually went in and examined that you'd followed these rules appropriately. and the insurance side mitigated the residual risk that will always be there when we introduce new powerful technology into society. We've seen this flywheel of standards, audits, and insurance time and time again when new technology has then been introduced in society.

7:05Emil Lassen:We see it again in cars. Cars have safety standards. These were not demanded by government. They came from industry themselves because they knew that if we develop safer cars, people are more likely to buy them. And safer cars actually enable you to drive even faster as well. So it was industry standards that also led us to airbags and seatbelts and some of the other things that now make cars safer. We naturally have, again, the third-party auditor going in and checking these cars and ensure that they follow the rules. So we have the inspection element again, and we also have the insurance element.

7:35Emil Lassen:And this flywheel, one of the best things about it is that it really scales. So we're not just seeing it with, say, light bulbs and cars. We also see it for nuclear power plants to this state where you also have standards inspections of those power plants and insurance even works in this case as well. So there's no limitation to the power of this flywheel. When we're looking at AI, we see some of the same things at play. We see a new technology that is very powerful and has the powers both do a lot of good, but also if things go wrong, can have severe financial implications. So, and the other thing is it's a complex industry where me as a startup saying my technology is safe, creates limited trust if I'm an enterprise buyer, a big bank, for example, that wants to adopt this technology.

8:25Emil Lassen:So with the artificial intelligence underwriting company, what we're trying to do is to create that trust layer in between the companies building AI and the companies adopting AI. And what we offer as the trust layer is this flywheel. So we go in and codify the standards we believe that the companies building AI should follow. We go in and audit against those standards in collaboration with third-party auditors like Shellman, Coalfire, companies who really know how to go deep and validate that the standards actually followed. And then we certify companies against the standards. A big part of the certification in the case of Agentsic AI is red teaming.

8:59Emil Lassen:So we go in and test the actual AI agent systems, not just to see that the policies they have are in place and work well, but that the agents actually work in a robust under pressure. And the companies that then obtain a certificate gets access to buy insurance of their agents so that there's also that financial coverage of residual risk.

9:17Daniel Whitenack:Yeah, this is so interesting. And I have so many questions, maybe one question that's just very selfish. And our listeners know some part of the joy of being able to do a podcast like this is I get to get my own questions answered by people that are smarter than me. But one of those questions that I have that actually comes up in conversations I have day today is this tension of, hey, I see a standard out there, whether it's some of the standards that we'll talk about that you all are codifying, or maybe it's things like the NIST AI risk management framework or things from OWASP. And logically, they say, yes, it would make sense to do those things.

9:58Daniel Whitenack:But what is the forcing function that is kind of making making companies consider actual implementation of those of those principles rather than having it be a be an aspirational thing? Is it the potential, you know, PR risk to the company? You mentioned the financial side, maybe it's the commercial side of getting, you know, software vendors getting their software into the hands of their enterprise customers. What do you see as some of those main forcing functions? Or are there even those forcing functions right now that would force people to consider this as something, you know, not aspirational, but actually practical?

10:42Emil Lassen:Yes, I see a couple of different things that I think are very practical. Any vendor building powerful AI right now knows how tricky it is to get through the enterprise vendor due diligence process and questionnaires. So these startups face these questionnaires. Sometimes there's 100 questions on them and it's extremely painful to go through. And I can tell you also, I speak to a lot of enterprise CISOs and NGST managers. It's equally painful on their side, right? because they're at a stage where the space changes so often that they feel the desire to actually change their questionnaire every month.

11:15Emil Lassen:Going through 100 questions from a startup every single time you try to onboard a vendor is also completely painful on the other side. So I think part one here is speed and a desire to get to a place where you actually feel like you've covered your blind spots as well. And having a third-party developed standard with all of the industry in the room to help find those blind spots and figure out how we can't find them in a standard, I think is value proposition number one. The speed argument obviously assumes that you can get across the line in the first place. I think the second part of the value proposition is having that third-party validation that your agentic AI is actually safe, secure, and reliable.

11:51Emil Lassen:We're working with some of the frontier companies in the AI space right now, companies like 11 Labs, companies like Fin that just got acquired for 3.6 billion by Salesforce, companies like UiPath who have set the standards within their categories historically. They have fantastic security postures, but they don't have a way to prove that to an enterprise. So an enterprise will just never trust a company that has an incentive to sell their product. They need that third party to go in and do that. And then I do think there is a security argument to be made here. Our red teaming consistently uncovers blind spots for the companies that we work with.

12:27Emil Lassen:Sometimes it's the hallucination rate where we realize that a specific type of adversarial attack will bring up the hallucination rate or specific language switches or other things that might actually happen when their products are deployed. Other times it's jailbreaking risk that we managed to uncover or pump injection risk that we managed to uncover. So we do also see ourselves as helping these companies actually improve their safety, security and reliability posture, which is valuable as well. Then I'm sure there's a marketing benefit to the companies going out early and adopting a new framework and showing and demonstrating they're moving in front when it comes to AI security leadership.

13:06Emil Lassen:I think that's an important branding value as well that we sometimes help provide, but I don't think that's a core benefit of the certification right now. I think that is really unlocking upmarket enterprise revenue for the companies.

13:21Daniel Whitenack:Agents are impacting every function within a company, but it's sometimes very difficult to figure out what an agent should do, what a human should do. Jeffrey from Noose Research, a recent guest, said often agents have no taste. That's why I'm so impressed with what our partner Framer is doing with their pro website builder that's already trusted by companies like Miro and Perplexity. They're implementing agents, but in a way that agents and humans work in tandem. Agents bring speed and scale, but people bring the taste, judgment, and control. And these agents help solve this gap between AI generated ideas and production ready website work.

14:04Daniel Whitenack:So Framer is already enterprise level solution. They allow you to create amazing websites that are SEO ready. And so I really would recommend that you check them out if you're building a new website or just implementing landing pages or an upgrade in your existing website. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.com slash practical AI for 30 % off a Framer Pro annual plan. That's framer.com slash practical AI for 30 % off framer.com slash practical AI. Rules and restrictions may apply. Yeah, yeah, I love your answer.

14:47Daniel Whitenack:And it was a little bit I was trying to validate some of my own thinking through that because we've talked on the show before about, you know, it isn't really like the the governments of the world are quite behind in terms of, you know, how they would, you know, enforce or even say what to enforce for companies building AI things. And so it's really the enterprises themselves that have some motivation to do this, all of those that you laid out, and I'm sure more. in in the in the current I guess state of AI standards if we kind of shift to that piece and then eventually I want to get to kind of the evidence and red teaming and all of that but maybe just as if we take a general look at the standards that exist out there for AI and AI agents could you help us understand what kinds of standards are out there and what they cover because there's a lot of sort of intersections that we could think of, whether that be security or safety or alignment or all sorts of things, data privacy.

15:57Daniel Whitenack:There's all sorts of ways that you could kind of look at this and perspectives that you could look at it from. And there's all sorts of things that people have proposed over time. So I imagine that's part of the reason why having a company that's really digging into this at a deep level is very worthwhile, which I think it is. But could you help set the stage for that? Like, how can we categorize in our mind the current state of AI standards and what perspectives are coming from?

16:25Emil Lassen:Yeah, absolutely. And by the way, Daniel, that's exactly where we started last summer, right? So if you go to AUC1.com today, you'll find that we've done crosswalks. I think it's about 10 different frameworks now. They're transparently available, so you can see exactly how our standard fits into the existing environment. And hopefully you'll also see then why we concluded after doing this work that yes there actually was a need for another standard even though there can be sometimes a little bit of standard fatigue and just uh by way of i don't

16:52Daniel Whitenack:know encouragement or thanks maybe gratitude is the right way to put it our our the company i laid has has looked at that many times in terms of and we're maybe in not not like everyone we're building actual you know control plane that works on some of these some of these knobs and levers that you talk about, but it's been extremely useful. And even, you know, as we're writing content or doing planning or thinking about things in our product, I always refer people back to that page and I'd refer our listeners to that page because it is a really great crosswalk and helps understand, you know, where these align, where they don't align, what the other need is.

17:31Daniel Whitenack:So just by way of gratitude, thank you for putting that together and making it public.

17:35Emil Lassen:Yeah, no, and we appreciate all the people who've worked on this. We do a lot of work with the Cloud Security Alliance, with the OWASP community across both the AI, VSS and the Gen AI project. We work with Cisco and IBM on Crosswalk. So it's a big team effort. And I really appreciate that we've been able to gather the ecosystem around a design to just publish some of this stuff transparently so that organizations like yours. But also I know big enterprises are using the controls we put out transparently in their own control frameworks. That's completely free to use. and only the companies pursuing certification actually needs to get money out the pocket.

18:09Emil Lassen:To get back to your question, because I think it's an important one, the way we see the standard space is that you have three layers. You have an organizational layer, you have an infrastructure layer, and then you have the agentic AI layer. At the organizational level, many organizations have been through an ISO 27001 certification, classic management system certification. ISO then about three years ago now published the 42001, which is the management system certification for AI systems. It's a governance certification that ensures that you have the right policies in place and the right, say, procedures in place so that when you develop AI systems, they hopefully turn out in the right way if you follow those systems.

18:47Emil Lassen:Then you have the infrastructure layer. That's where your SOC 2 comes in and your pen testing and some of the classic cybersecurity controls, access management, transport security, all the good stuff there. I'd say that many of those things become even more important in an agentic space because pace is higher, data access is higher. So if you don't have that in order, then you should go back and ensure that you get those boxes checked. And then at the agentic AI space, we basically just didn't see anything when we started this company and started drafting the first version of AI UC1. We see NIST have come out with the AI risk management framework.

19:25Emil Lassen:There's a little bit of agentic stuff in there. And I know from speaking to the team that they're considering publishing additions to this. The Cloud Security Alliance has also done their AI controls matrix, where again, there's some things in there around agentic AI that are pretty good. The issue with both of those frameworks is that they're guidance. They're voluntary frameworks. You decide which controls you implement. You decide whether you, like how you implement them. They're not orderable frameworks. So the way AUC1 fits in here is that we've basically taken the core governance things from the organizational level that we think are really important when it comes to AI systems, such as having failure plans in play when agents do not do what they're intended to do and you know how to deal with that.

20:09Emil Lassen:Good change management and acknowledging that every time you, for example, replace the LLM in an agent, it will behave differently. And if you don't take that into account in your governance, your end users will bear the burden of that. So some parts of the governance, The core parts of the infrastructure layer as well, so ensuring that the folks who have access to the AI system itself and can make these big decisions, that's restricted. Ensuring, again, that transport security when you do agent-to-agent communications and so forth is in place. But otherwise, we basically leave ISO and SUC2 to do what they're really good at and focus on the agentic layer.

20:44Emil Lassen:And what is up there then for us is specific controls around safety, for example, ensuring that agents behave according to brand and that they don't give users guidance on medical care or legal advice, financial advice, other high risk areas, basically that they stay within their scope and don't start breaking out of that. We look at specifically how you restrict the agent's data access, its system access and its tool access so it doesn't start processing refunds when you shouldn't. We look at hallucinations, which is also a risk that is quite unique to AI, obviously, and does not come up in any way in either ISO or SOC certifications.

21:21Emil Lassen:So really focus on the agentic layer there. And the core part of the differentiation, I'd say, is then the technical level of the controls. We go in and are actually quite prescriptive in what we want to see from the agents. because we have a good understanding now of what the right toolbox is to ensure that these agents behave safe, secure and reliably. And the other thing is we acknowledge that a technical control in itself might not hold up under robustness. So six of the 40 mandatory requirements in AOC1 have to do with red teaming, actually testing that these technical controls then hold up under pressure, both when we engage with the system as a benign user, just ask it questions and see if it hallucinates, but also what happens when we start approaching the system like with social engineering and adversarial pressure.

22:10Daniel Whitenack:And just to help people understand, so the AIUC1, that's the standard that you all have published. People can look at it online. I assume since it's AIUC1, there's an anticipation there might be a two or other or various, you know, either revisions or different focuses kind of within different certifications is my understanding right there.

22:34Emil Lassen:That's correct. I think where to start is we update the standard every single quarter. So we've gathered now a consortium of 250 security leaders. Some of them are CISOs at Fortune 1000 companies. Some of them are security engineers, architects, GSC managers. And so we have the full stack of people in the room. And with them, every quarter, we identify new priority areas. Last quarter, it was MCP risk, for example, which has really come up as agents start, not just operating in isolation, but exchanging information. This quarter, we look a lot at how we can strengthen runtime security and that continuous element, which continues to be really important for a lot of organizations.

23:13Emil Lassen:So we get them into the room and update the standard each quarter. I could very well see that new frameworks, so an AIC2 or AIC3, come out in the future. We don't have any plans to do that yet, but what we know, again, if we go all the way back to where we started our conversation, is that this combination of standards, audits, and insurance have worked historically. So right now we focus on the application layers of the platforms and products that take it into AI and deploy it. But there's a model layer as well, which we see as our second horizon. And there's the physical layer, like the data centers and the infrastructure that we deploy AI on, but also the cars and the robots that we put this into where standards, audits, and insurance could play a big role.

23:56Emil Lassen:And that's where we see the company go long term.