Empowering Compliance: Patronus AI's LLM Evaluation Tool for Regulated Industries

0:00In a significant move for the AI sector, Patronus AI has emerged from stealth mode today. So they've announced a$3 million seat funding round and unveiling its products designed to essentially evaluate and test large language models. So the startup is the brainchild of two seasoned AI experts, which are Rebecca Quain and Arnon Kanapin, both of whom previously kind of honed their skills at Meta. They were working over there. um quion focused on responsible language processing nlp i'm a researcher at meta ai while knappen contributed to the development of explainable machine learning frameworks at meta reality lab so patronus ai's timing seems almost serendipitous the startup aims to provide a security and analysis framework as a managed service and really they're kind of catering to specifically to kind of regulated industries where errors can result in considerable repercussions right?

0:54So one of the key areas that Patronus AI addresses is the likelihood of hallucinations in large language models. And that whole scenario where a model is going to just make something random up is what they're trying to solve for. So this is what they said. They said, quote, in our product, we really seek to automate and scale the full process and model evaluations to alert users when we identify issues. That was Rebecca Quain. And she also kind of elaborated the company's approach is three-pronged. So the initial step is scoring. So that is then followed by the generation of case text and finally benchmarking.

1:29So scoring assists users in assessing models based on criteria like hallucinations, especially in high stakes fields like finance or healthcare or the military, other areas like that, right? And subsequently, the system auto-generates adversarial test suites and performs stress tests on the model. So benchmarking the final step uses various metrics to determine the most suitable model for a specific task. So the startup is not just addressing the functional aspects of large language models, but they're also kind of looking at the ethical dimensions. They said, quote, we help companies make sure the large language models they're using are safe.

2:08We detect instances where their models produce business sensitive information and inappropriate outputs. So Knappen also noted, really really he just kind of stressed the importance of Patronus AI as an impartial third-party stating quote it's easy for someone to say their link their large language model is the best but there needs to be an unbiased independent perspective that's where we come in Patronus is the credible check mark this is something really interesting because in the past there is a lot of like obviously there's every AI model like you said is going to say like no we're fine we're good we have like we have been we have safeguards in place yada yada even open AI is like you know trying to put stuff in there where they're like no we got like a middle layer we got a trust and safety layer yada yada but it's like at the end of the day you can't really trust just a company's word for it when they say that they do everything perfect so i think having these third parties come in really is a good play and i think that's why a company like this um is going to be valuable um you know a lot of people are like whoa don't you think you're going to be like completely obsolete once open ai like just builds their own version of this it's like not really because how much do you trust every AI company?

3:14And maybe, right, maybe you are a good-hearted person that believes OpenAI is 100 % perfect at everything. Fantastic. But there's no way you can believe every AI company is good at everything. So this company, I think, really is going to be valuable in the future. It's going to exist for evaluating a lot of different large language models. So Patronus AI may not, or I think it actually does have six full-time employees at the moment, so not a ton. but it has plans for expansion on the horizon. When asked about the company's future hiring plans, the founders remained open-ended but emphasized the importance of having a really solid, diverse organization.

3:52And I think the$3 million seed funding is spearheaded by Lightspeed Venture Partners with contributions from Factorial Capital and other industry angels. With these resources, I think they're really kind of poised to navigate the intricate labyrinth. Like, let's be honest, this is an absolute labyrinth of challenges and opportunities that are ahead of this specific AI landscape, especially when they're addressing, you know, all of these fields that are typically a little bit more challenging. These are like really regulated industries and areas that would be hard. So it'd be interesting to see how they grow and adapt to those challenges.