Lawrence Snapp (TrustScale): Fighting AI Hallucination

0:09Kevin Werbach: Hi, I'm Kevin Werbach, Professor of Legal Studies and Business Ethics at the Wharton School of the University of Pennsylvania. For decades, I've studied emerging technologies from broadband to blockchain. Today, AI is promising to transform our world. But AI needs accountability, mechanisms to ensure it's developed and deployed in responsible, safe, and trustworthy ways. On this podcast, I speak with the experts leading the charge for accountable AI.

0:39Kevin Werbach: AI systems still make things up, and most people never check. My guest is Lawrence Snapp, CEO of TrustScale, which builds tools across the AI governance stack. We talk about what it looks like to verify AI outputs against evidence, why hallucinations won't go away, the prospects for a more robust AI assurance ecosystem, why he sees trustworthy AI as analogous to cybersecurity, and why he thinks AI users should pause more. Here's my conversation with Lawrence Knapp. Lawrence, pleasure to have you with me on the podcast. It's great to be here. First of all, why don't you give us some sense of the history of TrustScale as it grew out of another company called E2F.

1:23Kevin Werbach: Talk about that transition and what was the opportunity that the team saw? Yeah, Truskill is actually a DBA for ETAF. And so it was a rebranding. And in the spirit of some other things going on in the market, we could call it a refounding of a company. We're over 23 years old, founded by a French AI scientist who was going to graduate school in France in the 80s when it was kind of science fiction to study science, artificial intelligence science. but he was truly a language expert, an AI expert, and he got into the business doing French translation, quickly moved into chatbots, voice assistants, you can imagine the big three.

2:05And from there, a lot of those groups obviously were folded into the LLM data space. And so he really evolved very naturally, building his company, and he actually just retired earlier this year fully, but this was his opportunity to retire and for us to basically bring the company forward and really converge the latest on the technology and the human side to help AI become what we all want it to be.

2:33Kevin Werbach: So BGF was doing things like translation and data work. What was the shift with TrustScale and focusing on these AI assurance and governance issues? Yeah, I mean, essentially it was evolution in the industry. The company's been doing, we'll call it corpus data, call center, decision science in the early days, and then a lot of training and decision science systems for, you know, you can imagine the phone trees that we've all been stuck in. And then the early AI agents. But this really was this evolution where we followed the customers. And as the customers moved from, we'll call it simple data to much more advanced data and now obviously AI data, the company had a tremendous background, their founders experience in everything AI from the corpus pre-training to the post-training to the eval to the assurance.

3:28And obviously the hot topic is alignment right now. But he actually had a really deep experience in that. And so did the team. And as the buyers moved from voice assistance to LLM building, they took us with them. And we were very fortunate to go on that ride in the last few years as we've seen some great growth.

3:48Kevin Werbach: What attracted you to come on board? Yeah, listen, I think it's a tremendous opportunity to be at the frontier of possibly one of the biggest humanity transformations that has ever occurred. So on that side, you know, that's pretty exciting. The second part is I'm originally from San Jose. My internships were at Atari and Apple when I was in high school. And for me, you build great stuff. You do it responsibly and you hope you change the world. And if you do that stuff, usually success follows. And so I couldn't have seen a better opportunity to try to get on the playing field of AI. From my background, I'm also a CPA by trade.

4:28So auditing, assurance, controls, systems related to that is very much in my DNA.

4:37Kevin Werbach: So the term trustworthy AI gets thrown around a lot in different contexts. How do you define it, especially with relation to what you do at TrustScale? That's a great question. And I've got a very specific parallel. In the early days of cybersecurity, there was a memo from Bill Gates to Microsoft, where I'm an alum as well, called trustworthy computing. And you can still find it online. I think it was 2002. And it really was the emergence of the whole cybersecurity industry as we know it today. But it was about whether or not you could trust a PC. I think trustworthy AI is in the same exact camp today.

5:12We're at the very beginning of a curve of whether or not AI can be trusted. And it's really following the exact same curve, I think, as cybersecurity, although at a much faster pace. But it is kind of an amorphous term also, right? And that's what makes it so appealing. But when you break it down, there's a lot of elements. I think in the simple level where the world's at now, it's really about obviously safety, reliability, and there's a whole hierarchy, a ladder of rungs that we work on as far as evaluations and creating rubrics to define it. but where is it going is probably more interesting.

5:47And that's the nuance, the nuance of language. And, and the reality that what I say is maybe not what you hear and, and even what you heard might not be what I meant. And so there's a lot going on in the beautiful, messy humanity that we are that, that goes into this trustworthy AI opportunity.

6:09Kevin Werbach: So all the LLM still have these things on the box that say, you know, it's AI, you know, may don't trust it or may not be accurate, something like that. But you look at the technical reports that all the labs put out and they show hallucination rates dropping precipitously and being negligible numbers. So, you know, how much is this already a solved problem? Yeah, no, listen, I mean, the foundation of this whole thing we have to realize is that AI is probabilistic. It's a recommendation engine. And it's been around a while, spell checkers, grammar checkers and Microsoft Word. These things have been around.

6:45And while it certainly has improved, it's still a probabilistic technology. It's not deterministic and it's not conscious. But you specifically ask about the self-reported benchmarks of which there's a lot of, I call it the race of mules right now. We're not even in the horse race yet. Which mules ahead this week? And I think that end users are smart enough to know, and they've certainly experienced it themselves that there's a lot of issues left in the probabilistic technology that AI is, whether it's recency, whether it's accuracy, whether it's faithfulness to the context, or whether it's pragmatic.

7:21And it's not just semantics. Language and being able to trust the outputs of a probabilistic system require a lot more than did you pick the right next word or token. So I think that the foundation of this whole thing is that you can look at the benchmarks and especially self-reported, but the Foxwatch and the hen house is not how great businesses were built. And controls are necessary. Again, I think it's an amazing superpower. We believe in it very much, but in order for it to achieve that trustworthy North Star, there are a lot of harnesses and controls and systems that need to be built around it, hopefully not by the Foxwatch and the hen house.

8:03Okay.

8:03Kevin Werbach: But then if I pivot around to the other side, as you point out, this is a probabilistic technology. So is it then at some level an unsolvable problem that we can never be certain what outputs we're going to get? Yeah, that's a great question. I think this is a little bit like a limit. I think we're always going to be pursuing that trustworthiness. There are certain scenarios on the kind of simple questions where probably has exceeded the human ability to provide intelligence that is accurate. but we all know that, especially in the agentic world that we're in, you know, if something gets tripped up in the first agent or sub-agent, every subsequent agent is trained to essentially take that string and follow it and, um, and, and build on that confident hallucination that occurred.

8:48So you can actually end up, uh, in an exponentially different place than you thought you would, um, even though it feels good. And that's how these models are trained. So I, I think this is one of those never-ending journeys, and given the amount of petabytes of data every day that the world generates that needs to go into the AI corpus or post-training, that brings up another issue, because the amount of slot being generated in those petabytes every day is tremendous, which only makes the challenge harder to say, hey, was that piece of evidence we're comparing the output to actually trustworthy in and of itself?

9:25Kevin Werbach: What then does your company do to address to that challenge. All right. So TrustScale really works across the entire range of AI from working with data houses that have proprietary data sets to make sure there's hygiene and where there's redundancy or collisions that they're sorted out so that it can involve really good pre-training of the corpus data. And then we do a lot of post-training as well, not just on the verifiable rewards, but certainly on the preferences, as well as the supervised fine tuning. So in that post-training area, we do a lot of work, or you can imagine some of the top few market cap companies in the world.

10:05And then you move on and you say, okay, well, these rubrics that we created to define either behavior or preferences or outcomes or verifiable reliability, that needs to be evaluated as well. So we do evals on a lot of the models, And then you move it forward, we're actually starting to work with some of my former industry colleagues in the audit and assurance world, looking at how they, as a third party, can have tool sets that they can do their independent verification, especially now as regulations are starting to come on board to protect consumers and businesses from the harms. So TrustScale itself works across all of that.

10:44We've got products in each one of those stages of the value chain. But at the end of the day, what we're doing in a simple way is taking probabilistic outcomes and comparing it to deterministic control plane.

10:57Kevin Werbach: And that gap analysis is not simple. This isn't just the baseball score. It's nuanced. And in fact, we call our business of specifications and rubric creation nuanced for that reason. Yeah. How do you deal with a situation where there's not a clear factual answer? I think that actually the majority of things in life are not factual. In fact, we don't call ourselves truth scale. We call it trust scale for a reason. Trust scale is because we think that it's up to the individual to understand what their truth is. You might love peanut butter. It might kill me. We're both right. Those are truths. we see a lot of things, especially in the media today, where there's three unemployment rates and every politician is using the one most favored to themselves, whether it was the whisper number, the first reported or the corrected number.

11:49Well, that's a good example that all three numbers are true, but how do you trust it? So our business is more about empowering humans to make the judgment call or giving a complete and accurate answer when there is nuance or what I call the beauty of the messiness of humanity, the yellow. Red and green is real easy, but the yellow is where the magic is.

12:12Kevin Werbach: Do you have to work with the customers to get them to be able to articulate effectively what their preferences are? Absolutely. We spend a tremendous amount of time, and this is where we have a lot of in-house experience and systems for doing it around the specification, the rubrics to measure things, and the quality frameworks. I mean, some of these are 100-page long efforts that go into our systems that then can help us actually determine, you know, not just what the right or wrong answer is, but when it's not clearly right or wrong, what do you do with it? How do you train the model? And humans are really complicated things.

12:54A lot of people will get excited and say, well, that fact is wrong. Well, we're more in the empowerment business, I would argue, than the truth-telling business. And so our job is to illuminate these nuanced differences and let our customers decide how that wants to manifest. And hopefully it's in a complete and accurate way so that the end users or the agents are fully informed. Because that's where I think a lot of the world gets in trouble. What do you mean by that? Well, it would be very easy to tell you that the unemployment rate was 4 % because I heard a whisper number. That doesn't mean it's a complete and accurate answer if that's what you're asking, because the actual number might have been 3.7 and then the corrected number might have been 3.2, right?

13:36That really changes a lot of judgment. So I think that empowering people with complete and accurate information is a much better road to trust than than trying to say, here's the truth. And I think, as we all know, whether, you know, in any relationship and, you know, in just in life, especially if you have kids, you know, their truth and my truth might be different. And that's OK. And that's where, to be honest, Kevin, it's really exciting. But the future is where we literally have this kind of mass customized or individual perspective of AI so that each person can configure the AI to the way that they would trust it most.

14:16Kevin Werbach: if we go down that route though you mentioned before uh auditing and and your background and there's a lot of discussion about moving to you know evaluations to audits for ai is that feasible in a world though where it's not there's not a right answer it's this uh decision based on what's useful for a particular audience yeah absolutely but i think a good example um choose the unemployment again, if you're auditing the AI output, you have to wonder about the integrity of the system if it's not disclosing the full or complete and accurate information. So completeness would say, hey, you know what?

14:53You told the agent it was 4 % when it was 4.2. Why? Because you picked the convenient number or because you're trying to make them feel good? So I think that audit reports would be much more complex than when you have an empirical financial system that you're auditing. Right. And even those audits though, right? If you read the opinion letters to these big firms, it's always about material correctness. I think it's going to be tough to go beyond materiality, but in this case, you know, language is nuanced and humans want to be empowered with more information. And so I think we're going to see very different auto reports in the future.

15:26In fact, we're, we're developing systems right now that, uh, interpret things, um, in the yellow, you know, The red and green is easy, but when it's not clear red or green, that's where the magic is. That's where we spend most of our time.

15:39Kevin Werbach: What do you think are the biggest gaps today to having a more extensive and broadly functioning AI audit ecosystem? Well, the AI audit ecosystem in general is just so nascent, and I think part of it is education. You know, it's people like you getting the word out of the necessity for organizations to be developed that do things in partnership with their customers or their clients, but always understanding their North Stars is, I think, an AI about humanity, not just profits, because this is a little bit bigger than just financial metrics. AI has this superpower potential, and we're already seeing little pieces of it here and there.

16:22But to unleash this without continuous monitoring, without proper interpretation and reporting and observability on the nuances in between right and wrong, it's going to be difficult. So there's a lot of education to be the first and foremost thing we need to continue to hammer. Then from there, obviously, there will be generally accepted frameworks that will be applied. And I only hope that they're really comprehensive because if we're trying to basically mirror humans and neuroscience, we got a long way to go before we get there.

16:55Kevin Werbach: So let's say I'm an enterprise that has an AI governance program, which is all of them. Help me understand where I'm going to most look to trust you. You talked about how you work all the way across the staff, but are there particular problems that you're really good at solving for them? Yeah. You know, some people refer to us as an applied governance company, and I think the keyword is applied. So it is difficult to really try to reach that holy grail of trustworthiness if you're not looking at the entire value chain of what you're creating. And these days, and you talk about enterprise, there's a lot of talk about sovereign AI.

17:29So if you've got your own private corpus, you've got a public corpus. are you um conditioning with hygiene that data before it starts and we we help with that then post training preferences are you giving complete and accurate answers or the shortest ones because you want to save server power well we can help with that and we can make sure that you strike that balance of value for your organization then you move forward on assurance internal audit groups some organizations that we work with we have tools that we can empower our internal audit groups with. And that's a really exciting and emerging area as well.

18:04So we love that one. And then the one that's most visible is what we call Artis. Externally, we released a Chrome extension and you can go to trustartis.ai. What's exciting about Trust Artis is it essentially real-time compares the outputs of AI and gives the user red, yellow, green, or highlighting with the opportunity to correct or adjust the outputs Should we find evidence inconsistent to some degree or a great degree with what the AI generated? So the most visible thing we have for just people is go to Trust Artist and you can try it. It's really fun on social media and the media, CNN and Fox and those companies too, because it really illuminates where there might be contradictory information of that.

18:49Kevin Werbach: How would a company use that? I mean, I understand that as a demo that you can use it as a customer, but if you're an organization, you've got lots of users, lots of AI you're developing, how would you actually create the feedback loop to implement what that tells you? Well, there's two points in your question there. The first one is just end users, employees. They plug it in, whether it's their Chrome browser, Island browser, or they actually use our chatbot, Trust Artists, and we adopt that for enterprises. And then that gives a certain layer of protection, right? because it detects, it protects through the red, yellow, green highlighting, and it empowers people with evidence so that they can correct the AI outputs before they're used in a harmful way.

19:28So I think it's an exciting tool from an end user perspective, but then you asked about the loop. The loop is so valuable and that's where the post-training opportunity is. So everything that's captured in an organization can be looped back and used to try to prevent that same hallucination or idiosyncrasy from happening again. And so there's two sides of that coin on Argus, the one that end users see to protect them and to empower them to correct problems before they cause harm. And then the administrative side, which is those people actually making or shaping AI or creating agents to use as input for their training.

20:09Kevin Werbach: And can you drill down a little bit more to how you determine what's red or yellow? Presumably, again, the factual issues where it says something and you can look at ground truth, that's pretty straightforward. But as we talked about, that's not most things. So how do you address it? Yeah. I mean, listen, think of this as a real-time, high-powered research tool. It takes an atomic-level claim, a semantic level, and it will find similar information elsewhere with only about a 15-minute delay worldwide on information. And then it will compare it. And then it can get multiple pieces of evidence and enterprises can adopt their own guidelines if they want.

20:47Let's just say there's three pieces of evidence. Two are consistent. One is not. Some organizations in certain environments say, hey, that's red. I don't even want to risk it being yellow. Other organizations, and we'll take a creative department of Disney, right? They can just say, hey, listen, if two out of three are saying that that's consistent with other evidence pieces we find that's trustworthy, hopefully, then that's green. That's fine, right? So it really depends on the domain. And from an enterprise perspective, Argus is completely configurable. And it's exciting. From an individual perspective, it is not at this point configurable.

21:24And we essentially, you know, if you take three pieces of evidence, if all three are consistent, that's green. You know, one or two aren't. It's probably yellow unless it's a real obvious a breach. And then if all three are wrong, of course that's red. But the key is that we give the links to the user to be able to do the research themselves and then determine for themselves. Again, we want to empower the individual, not take it away from them. And so our tools right now, especially in the public facing, stop short of actually just overwriting the AI's response, which we could do. And that's because we really believe trust is about empowerment and knowledge.

22:04And so we're excited about how it's working and we're excited about the feedback of it so far as well.

22:10Kevin Werbach: Although I assume there's a challenge there because on the one hand, if you push that back to the user, then there's some additional friction in the process and some additional time the user has to invest. But presumably if you're in the background rewriting things, you have a trust issue about what you're doing and the user then is no longer having to engage in that cognitive activity. So how do you find the right balance? Well, listen, I absolutely love you going down this path because one of our big tenants as a company is to make you pause. Like we really believe that our job is to help make you pause.

22:47Why? Because this cognitive surrender has just been consuming, especially young people, students. And listen, we all went through with social media where it's pretty easy to get in, you know, a rabbit hole and short horn video or whatever. It was cognitively surrendering. Oh my God, where was I going? But with AI, it's really dangerous and much more nuanced. And these systems are trained to give you good vibes and, and to actually the more it knows it's hallucinating, make you feel it's even more confident in the response. Those are dangerous things. So we like, um, that it prevents cognitive surrender by asking the user to get engaged with what they're seeing in a red and a yellow.

23:27Those are stop signs and yield signs. And then empowering people with data, yeah, we are putting a little burden on them. But I think this is why we're seeing so much in law schools, healthcare, people that are in domains where there's real consequences if you trust a hallucination blindness. But this is the early stages, I think, of a hundred-year transformation of humanity. And so we're hoping to make people pause, avoid cognitive surrender, especially unconsciously, and then get the brain neurons firing again.

23:59Kevin Werbach: Yeah, and you started to get to what I was going to ask you next, which is where are you seeing the most interest in demand? What kinds of companies? Yeah, so obviously the higher risk domains. I think that there's plenty in law and healthcare, but academia across the board, every discipline. And then you move down, I think that the other thing is the content houses that have repositories of private data that are trying to figure out like, hey, I don't want to sell this to an AI frontier lab and then have my data show up as mine, but misinterpret it or taken out of contests or not pragmatic relative to the bigger picture.

24:35Because data alone is insufficient. It is about the semantics, the contextual, and the pragmatic level. And all of it needs to be considered. And that's what our trust skill engine does behind Argus on the visible side and Argus for the enterprise.

24:53Kevin Werbach: And what happens about this idea of the user taking control and pausing when we're in a world of agents and the actions are often not taken by humans? Yeah. Well, this is where it trips into what we have is really a cybersecurity product, right? Because these nuances, hallucinations, or some people like to call them confabulations, these are threat vectors. And one of the big things is that when an agent is integrated with Argus, it can essentially force the agent to pause and get a human to say, hey, should I keep going or not? And so we think that this is a cybersecurity product. We think it's a hallucination firewall.

25:33These are terms that our customers use when describing what they're doing with our product. And we're excited about that. So I think that the autonomous agent world, there's going to have to be gates. And there's got to be systems to put those gates up because no human can keep up with the volume that AI is producing these days.

25:57Kevin Werbach: It's interesting you frame it that way because, I mean, I think of the cybersecurity, there's an attacker out there. And certainly we have prompt injection and data poisoning and things like that where there's some deliberate act that's causing the incorrect information with the AI. But if it's hallucination of something internally coming in the model, it seems to me like that's different. Well, that's a really, really good point. Um, these models are trained to behave a certain way, to make the user feel a certain way. And when it can't get the confidence interval to know pretty certainly what the next word should be, they're trained.

26:36Some models are trained to flat out just make it up. Some are trained to say, hey, I'm not exactly sure, check this. But that's rare right now because that would really get in the way of people paying for it. Uh, and then there's, there's others where, um, you know, we're seeing, you know, 70, 80%, according to one Stanford study, uh, hallucination rates on, on like a legal dummy, uh, certain models. So it's all over the map, but these things are trained to give you good vibes and hallucinate deliberately. Again, it's a recommendation engine. It's not a deterministic outcome. and we need to always kind of consider that, but it's very easy to lull ourselves to sleep and just use the superpowers.

27:19Kevin Werbach: Yeah, I guess what I was wondering about is it's, you know, just the framing of it. And it's interesting if you're getting positive responses from customers on talking about in cybersecurity terms, just, you know, still that's not a situation where the AI company is acting in a deliberately hostile way. As you say, it's intrinsic in the model and it's in line with their business interests. But, you know, I'm just curious and interested about how that seems to resonate, the talking about cybersecurity. Yeah, so listen, hallucinations, depending on the domain, are threat vectors. They can kill people.

27:58They can hurt people. They can harm people. They can harm communities. These things are real, and there's a lot of gravity. And luckily, and obviously more recently with some of the press, there's been some more seriousness around it. But the problem is that, you know, we know there's needles in this haystack. We don't know what they look like. And, you know, as a human, you'd have to check every word to see if that's where the needle was. Well, that's just not possible. But those needles can hurt. And that's why this cybersecurity framing is just kind of became natural for us. And our customers and our prospects look at it and say, wow, I could sure use some hallucination protection.

28:38Now there's others that call this, well, these are just confabulations like humans do. And that's a softening of it. Personally, I don't subscribe to that. And I'm not saying that every employee is perfect or remembers things perfectly. And yes, we as humans confabulate, that's for sure. We might each remember this discussion slightly different. That's normal. But these models, when they're pulling data and then confabulating or hallucinating in high-risk domains, that's something that needs to be taken very seriously. And that's where we're excited that our services and solutions really can help protect, detect, protect, and correct the problems before they cause harm.

29:21Kevin Werbach: Right now, there's a lot of attention being paid to the catastrophic risks potentially of the frontier models. As you say, when you're talking about hallucinations, there can be very serious consequences, including loss of life. But it's not at the scale of these kinds of systemic risks that we're talking about. Um, you know, do you feel like all of the attention being paid to these potential existential threats is, is distracting people from this other domain that you're involved with? Well, it's interesting. Um, I would argue that even some of these existential threats, uh, that are, that are headlining the last few weeks, um, might start as a simple, small hallucination.

30:06and and so you can't quite ignore um the little stuff if you're worried about the big stuff now i think taking the knob and turning it all the way right is a great way to get pr and the mass audience globally to care and action hopefully but i'll tell you we have seen more times than not um one small hallucination have a ripple effect that cascades to create exponential drift or variance from the objective originally. And so you have to go to first principles of the root. Like, why did that happen and where did that happen? Well, more often than not, if you peel the onion back and you find the initial source, the core of that, it started with something small that became very big, very fast.

30:49And these are the existential threats, right? The bot's gone wild. And so there's a whole host of opportunity for companies like ours to figure out how to protect and detect and project against the little stuff so it doesn't become big stuff. But the big players, you can flash the shiny object of, oh, we're all going to kill ourselves, and you get plenty of headlines. But I think that it's really hard to say that one problem is any greater or less than the next.

31:22Kevin Werbach: Absolutely. Lawrence, thank you so much for your time. I agree. There's lots of pieces here, and I appreciated the opportunity to hear about what TrustScale is doing. Kevin, thank you very much. This has been The Road to Accountable AI. If you like what you're hearing, please give us a good review and check out my substack for more insights on AI accountability. Thank you for listening.

31:46Kevin Werbach: If you want to go deeper on AI governance, trust, and responsibility with me and other distinguished faculty of the World's Top Business School, sign up for the next cohort of Wharton's Strategies for Accountable AI Online Executive Education Program. Featuring live interaction with faculty, expert interviews, and custom-designed asynchronous content. Join fellow business leaders to learn valuable skills you can put to work in your organization. Visit execed.wharton.upenn.edu slash ACAI for full details. I hope to see you there.

32:24Thank you.