Performance AI: Insights from Arthur's Adam Wenchel – Ep. 221

0:10Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. We're recording at GTC 2024 in San Jose, California, and my guest for this episode is Adam Wenchel, co-founder and CEO of Arthur AI. Arthur calls itself the AI performance company, and they work with enterprise teams to monitor, measure, and improve machine learning models for better results across accuracy, explainability, and fairness. Adam's here to talk about Arthur's mission, and we're going to get into bias and observability in AI, guardrails on generative AI systems, and more generally about the adoption of generative AI in the enterprise.

0:48So let's jump into it. Adam, thanks for taking time out of your GTC experience to join the AI podcast. Yeah, thanks, Neil. I appreciate you having me on. So we were talking offline. You just got in. I was going to ask you how GTC's been for you so far. Did you get a chance to check anything out yet or not yet? Yeah, I did. I went to the session earlier today on the head of research, you know, talking about some of the stuff they're working on, which is always fun. So it's good to, you know, I started my career as an actual hands-on dev back in the day. So every once in a while, I like to pretend like I still, you know, follow the tech and get to enjoy some technical content.

1:21Walk around, pick up some buzzwords, not knowingly. Excellent. Love it. Well, let's talk about the hair now. Let's talk about Arthur. Maybe you can just start by telling the audience what Arthur AI is all about. Yeah, absolutely. So at Arthur, what our goal is, is to help people translate all the promise of AI into the real world, right? And so, you know, we've all played around with ChatGPT and Anthropic and Gemini and a bunch of other open source models. And really exciting to use and play around with, very compelling. But when you go to deploy systems based on these elements in the real world, you quickly discover there's some risks and some challenges that need to be managed.

1:59That can be hallucinations. It can be leaking sensitive data. It can be inappropriate use of the LLMs. There's a number of things. And just flat out performance, right? Are they doing a good job answering the questions? Are they value aligned with your organization? Things like that. And so that's where we help with. And so we started out as providing monitoring tools for models and AI, and then we've since expanded into, you know, that Shield, which is our firewall for AI, which really allows you to apply policies around the usage of these things and also just make sure that they're performing well.

2:31So what kinds of use cases are you seeing out there in the real world? You know, what do companies, what enterprises want to use LLMs and Gen AI for? Yeah, I mean, there's obviously been a lot, but I think a lot of established businesses are really, they're starting with internal use cases just because they want to really make sure they understand the technology and know how to deploy it responsibly before they put it in front of their customers and their partners and things like that. We won't name names. We've seen some chatbots go awry publicly. We've seen a few of those, yeah. And there's no question, like, there's a lot of value to be created by having a public-facing.

3:04But what we're seeing, like, first deployments are a lot more around internal use cases around things like HR, for instance, is a surprisingly popular one. It turns out if you're a large company, answering questions about benefits is actually a really time-consuming task. It is, yeah. And so there's, like, huge productivity gains. And also, a lot of times you get better answers that way. Right. So it increases kind of satisfaction among your team. And so it's things like that, you know, these back office tasks that are not glamorous, but actually are quite impactful that people are using. And, you know, you're seeing a lot like legal and investment, a lot of, you know, private equity and investment banks.

3:40And, you know, for those, a lot of those, you know, the price of an incorrect answer is quite high. Like if you give someone bad information about their benefits or if you give them, you know, or if you make an investment based on bad advice, you know, even if it's right 95 % of the time, 5 % is not great. Yeah, right, right. It's easy to kind of get caught up in the tech world bubble, so to speak, right? And I've been, you know, the minute a new update comes out, I'm going online trying to see where I can play with it and everything. But have you been experiencing, you know, working with customer companies who have heard about AI, have heard about LLMs, that sort of thing, but then when you get into actually implementing, like, it's all brand new to them?

4:22Or what's kind of the, you know, what's kind of the delta between what's going on at some, you know, place like GTC where we're at and then where sort of the rest of the world is with using this stuff? Yeah, good question. I mean, certainly the audience at GTC is on the tip of the spear of this stuff. But, you know, what I'd say is like a lot of even a lot of traditional organizations for the last four or five years have been trying to build up their capability around AI. and they've been adding talent and putting in place infrastructure, getting data ready, things like that. And it was kind of a very slow rise up the maturity curve, I would say, for the last four or five years.

4:56But generative AI has definitely caused an inflection in terms of the adoption, because for a few reasons. One, I think part of it is all that investment from the last four or five years kind of prepared people for it. But also because this has become, you know, a board member or a CEO can easily go on to Chash EBT and like, you know, it's so accessible And it's really easy to make the mental leap from, you know, you asking it to plan your weekend in San Jose to from that to like having it, you know, plan trips for work or for answering HR questions and things like that. And so everyone sees the possibility.

5:28And so there's a real sense of urgency where it is a board level issue. Like, what is our generative AI strategy? And there's some really strong imperatives that are getting handed down. Like by, you know, next quarter, you'll have like deployed generative applications. Right. Right. And is the motivation for that, like, are there specific motivations? Is it, you know, in the name of efficiency or is it more of a everybody sees where, you know, the puck is headed and so they're skating in that direction kind of thing? Yeah, a lot of it is efficiency, but not necessarily to cut costs. I think the idea is like if you can be more efficient, you can allocate more resources to like more strategic work.

6:04You know, if you can remove some of the tedium and some of the kind of more rote work, it frees up those resources to take on more strategic work. And so we haven't seen any evidence of like job loss or anything like that or even like significant. The cost savings all gets kind of reapplied towards, you know, going after more top line growth. Right. So in terms of the things that can go wrong and that, you know, with good reason, as you mentioned, companies are kind of deploying the stuff internally first so that the mistakes are a little easier to contain, that kind of thing. Hallucinations is obviously the thing that a lot of people have heard about.

6:36And, you know, an LLM making something up, passing it off confidently. Is that at the top of the list for a lot of your companies and the things that they're worried about or what other things are there? Yeah, I mean, hallucinations are really universal. I mean, wrong answers are bad for everyone. I think that, you know, if you have an online cocktail recipe generator, you can probably live with a bad drink. But if you're in any sort of business application, then wrong answers are, you know, whether you're giving like the HR application or investment advice or legal advice, you can't live with wrong answers.

7:09But there's a bunch beyond that, you know, prompt injection, inappropriate use of the system. So if you're using your HR system to try to generate legal briefs, that's not good. It's not good for a number of reasons. And in things like toxicity, you know, I think a lot of the universal notions of toxicity, the LLM makers have actually done a really good job of RLHFing, kind of some of the really obvious behaviors they picked up off the text, the internet text they were trained off of. But, you know, there's a lot of times use case sort of specific, inappropriate language. And so an example would be if you have an LLM agent you're using in the recruiting process or the hiring process, you shouldn't be talking to it about like family status or it should not be telling you about family status.

7:52And so I think there's sort of, you know, kind of use case specific policies that you need to be able to apply to make sure that, you know, you're not being reckless with the way you deploy it. Sure. And so kind of from more of a technical standpoint or sort of a pragmatic standpoint, I guess, are you fine-tuning, you know, models that people would have heard of and fine-tuning them for specific deployment? Are you training new models? You know, what's kind of the adoption process that these companies are using? Yeah, absolutely. I think people typically start with an off-the-shelf model because they're so easy to adopt, especially with some of the ones that are provided by the big names that provide them via API access.

8:33But we are seeing a lot of people train and fine-tune models for their purposes because you can get really, especially if your task is not a GP, not a general purpose, but is a little more constrained, you can fine-tune a smaller 7B model or even smaller on a particular domain. And you can have it give really good answers much faster and much less GPU, like much more efficient use of GPUs. And so we're seeing a lot more of that. We actually use that technique internally. some of the ways people can apply policies is actually using LLMs to evaluate the outputs of other LLMs. But, you know, you want to make sure that's efficient.

9:08You don't want to add latency to, there's already some latency. You don't want to add more to it, things like that. And so we use that technique ourselves and it's pretty effective. And I think people are really just starting to get good at it. Yeah. Let's talk about observable AI. What does that term mean? And sort of in practical terms, what does it mean? Yeah. So I, you know, my previous role before starting Arthur, I led the AI team at a large top 10 U.S. bank. And there, like, if you're deploying AI in any sort of mission-critical sensitive application, like deciding who gets a credit card and how much credit they get, then you need to be able to tell people, like, why were different people turned down?

9:45And you need to also make sure that they're making good decisions. And so the models are making good decisions. And, you know, in the old days, there's all sorts of stories about people deploying models. And they used to monitor them or observe them, it was a very manual process. Like once a quarter, some data scientists would go kind of check in, download all the data to their laptop, put it in a spreadsheet or a notebook and check it out. But, you know, nowadays these models are so dynamic and sort of the rate of change in the world is often so dynamic that you can't just sort of set it and forget it with these models.

10:14You need to bring that same kind of real-time intelligence to the monitoring and oversight that you have for the model itself. These aren't just, you know, the simple linear regression models from 20 years ago. Right, right, right. Yeah. How reliable is kind of the automated observation process? Is it something you're happy with? Is it something that's kind of a work in progress? Is it something that's kind of a hurdle that you're working to get over? Like, what's the seat of the art with observation? The observability is actually quite good. That part's quite mature. And I think that the maturity around a lot of, you know, obviously, in the last year, all the stuff around hallucination prevention and detection and, And, you know, there's a whole new suite of metrics that the industry's developed and we've helped driven some of that development around, which is like measuring things like readability or helpfulness.

11:01You know, how do you kind of measure generated text for like, you know, the equivalent of accuracy? There's a number of different axes you want to be able to evaluate your performance on. And so like the notion of like what is good performance, I think people have like, you know, we can go read stuff manually and sort of say whether it's good or bad. But how do you break that down into something that's measurable at scale? So if you're asking, you know, if you're answering like hundreds of thousands of questions or serving hundreds of thousands of users, how do you, you know, really look at that at a large scale and make sure it's performing well?

11:33It's a fascinating topic and something we like to talk about. Yeah. I don't know if this is actually a logical train of thought or not, but it makes me think about bias in AI systems. Can you talk a little bit about that? what, you know, it's a term that gets thrown around a lot. And I tend to think of it, or maybe I've read things in the media that make me think of it in terms of demographics, right? Racial bias, ethnic bias, gender bias, that sort of thing. What does bias in an AI system mean to you? And I think kind of more interestingly, maybe, how do you deal with bias without, you know, compromising the accuracy or introducing hallucinations or otherwise kind of messing up the AI systems?

12:13Yeah, it's a good question. So, you know, the bias that's in systems a lot of times is, you know, at some level, like the credit card example, right, or auto loaned, you know, people used to kind of manually, like you'd fill out an application in front of someone at the auto dealership, and then they would kind of be, it was a very subjective decision about whether to give you an auto loan. And there was certainly a lot of bias in those decisions. And then what happens is they took that data, like a decade of that data, and then trained models on, right? And so all it does is automate the bias that was already there.

12:43You train a model on the internet, you're going to get some bad words. You get some crazy stuff. Yeah, absolutely. And so a lot of it is being able to measure that. And so in traditional AI, there's dozens of metrics around fairness, and you can't solve for them. Actually, a lot of them are like, there's like tension between them. But roughly speaking, there's kind of a couple big, there's like two big categories. One is, are they having equal access to the outcomes? And so like, are they getting approved for auto loans as frequently or in the same amount? And then the second family is a lot around accuracy, right?

13:14Like, are you making really accurate decisions for males, but like females, you're just throwing darts at a dartboard and you don't really, your model doesn't really know what it's doing. So maybe they're getting auto loans at the same rate, but, you know, you're having a lot more repossessions with one group than the other because your model's not accurate there. And so that was traditionally AI, and that science is relatively well understood, although still, I think a lot of people are still learning how to kind of do it in the real world. But certainly from an academic perspective, I think it's fairly far along, and we've published a lot of papers in that space.

13:45At LLMs, it's a little bit different, you know, and it's actually, it's another area where people are just learning how to measure it. I will say that when generative AI first exploded towards the end of 2022, the first models that were made available were horrendously biased. And it was like trivial to get them to say things that no organization would want a piece of technology they're putting out there to say. And I would say that the model makers in general have gotten really good at softening the most egregious parts of that through, you know, RLHF and constitutional AI and things like that, policies.

14:18And so a lot of good work has been done there in the last year, and they're much less biased than they used to be. And so now the forms of bias, it still exists, but it's a lot more subtle. And so I think there's some really interesting research on how to, you know, make sure that the answers, you know, answers about questions, whether they're given in the context of a male or a female or people of different racial groups are pretty even and equitable and values aligned with the organization publishing them. I'm speaking with Adam Wenchel. Adam is the co-founder and CEO of Arthur AI, and we've been talking all things generative AI when it comes to deploying this technology out in the business world, in the real world, so to speak, across different industries and lines of work.

15:00You mentioned your own background working at a bank, and I wanted to ask you kind of how you got to the point you're at now with founding Arthur. If I've got this right, you started a cybersecurity company. at some point back in the day. Can you tell us a little bit about that, how you leveraged machine learning, and then maybe how your path kind of progressed to here? Yeah. So actually, I studied AI in school at the University of Maryland, which has a really good AI program, over 20 years ago. And at that time, it was not really in style. It was actually like mostly, you know, people would say jokes like, oh, you're in AI, you're great at predicting the past and things like that.

15:40But then I followed one of my professors was over to DARPA and worked on a couple of really early AI projects there, including one that was all about using agents to plan, which is, which has been, so it's been fun to see in the last year, like a lot of that AI stuff coming back and it's closer to working now, a lot closer to working now than it was back then. But yeah, I, I've then went from the, from DARPA into the startup world and I've definitely kind of been, been hooked by it. It's a, you know, just the, the creativity of like starting a new company and, and building it is just really fun.

16:09And like you said, my last company was using machine learning for cybersecurity, which nowadays, like every cybersecurity company says they're using AI or machine learning. But this was back in, you know, 2013, and it was a little more novel back then. Right, right, right. And then that one was acquired by Capital One, actually. So I joined Capital One, and they asked me to start their AI team shortly after I joined, which I did, which was a lot of fun. Yeah, I bet. Are there learnings from, you know, the DARPA days and the original cybersecurity company you mentioned, you know, Are there learnings from those days when, not that you were doing theoretical work only, obviously, but the compute wasn't there, things weren't as advanced as they are now.

16:49Kind of things that maybe you wanted to do or could only do to a certain extent that you're now able to kind of bring back and more fully flesh out? Or has time just kind of moved on and it's new stuff? No, I mean, certainly from the DARPA days, the compute definitely wasn't there. And we, you know, we weren't, we weren't using GPUs and, and you didn't have cloud computing, so you couldn't just spin up, you know, a hundred boxes to train a model on the data that you had much smaller data sets, which was holding it back. And there are also a lot of, you know, a lot of algorithmic improvements that have been made in the intervening 20 years.

17:21And so back then, a lot of the conversation actually, you know, learners, which are what we, what's in vogue currently, were a little bit out of vogue back then. And I actually liked him, but more people were, there was a lot more energy going into symbolic AI, which works, you know, it's very, it can be very brittle, but it's more compute efficient usually. And this is all generalities, but often. And so that was like kind of the, where a lot of the energy in the research community was going. And symbolic AI actually, at times it kind of sticks its head up every now and again to kind of address some of the limitations for learners, but we'll see.

17:52Yeah. Thinking about cybersecurity and, you know, LLMs and generative AI, and when you were talking about some of Arthur's company or client companies, you know, deploying internally first with good reason, it made me think of stories I've read about, you know, companies not allowing, I mean, especially when when ChatGPT kind of first exploded into the consciousness, you know, schools banned it outright, right? Lots of giant school districts. And, you know, companies would hear stories once in a while about, you know, some big tech company not letting their employees use it or spinning up an internal model for employees to use.

18:31And a lot of the rationale, at least that I heard, was, you know, fear of data leaks and fear of, you know, well, if I'm putting proprietary information up into one of these publicly available models, I don't know what's going to happen with it. Like, best case, it gets used to train their next model, which isn't really cool with us. Worst case, you know, it somehow shows up in somebody else's chat. Which happened. Right, right. Absolutely. So, you know, with your cybersecurity background, like, what's your take on Gen AI, LLMs, everything that's happening now, and the cybersecurity implications?

19:06Yeah, so, you know, when the first batch of models were stood up and kind of made available, people started adopting, like, all the things you mentioned weren't just theoretical concerns. Yeah, no, they happen. So, right. And so I think those companies have been rightly taken to task about that. And they've put a lot of energy into providing much stronger guarantees around that stuff. But, you know, depending on the sensitivity of your information, you know, the question is how much do you trust that? And that's why it's nice that there's, you know, you have optionality. You can run your own open source model in your own, you know, in your own environment.

19:37And if you're dealing with really sensitive data, like we have hedge funds that are very sensitive about their data. and they prefer to do it. And I don't think it's, it's not unfounded given what's happened. And then there's also a whole set of concerns around, you know, if these models have been trained on data that, you know, is proprietary, that someone else owns the rights to, and I'm using it to generate like my strategy or my investment decisions on essentially like someone else's proprietary, a model that's been trained on someone else's proprietary data, what are the legal implications of that, right?

20:08And so I think - Do you know? Do you have a good answer? Well, no one knows right now. And so it's the kind of thing where it's going to take years of case law to sort that all out. And so a lot of, you know, lawyers being lawyers are going to make sure that everyone's like, you know, taking the risk very seriously. And so we'll see. I think there's, you know, there's some, I've heard some groups are offering, you know, indemnification to their best customers for that and things like that. But, you know, I think, I don't think you can put this, you know, at this point the toothpaste is out of the tube.

20:38I don't think it's going back in. Yeah. No, what's the phrase? you know, do it now and ask for forgiveness later came to mind as I was listening to you. There's been a lot of that going on last year. Yeah. With all the different companies that Arthur works with and, you know, the different use cases, you mentioned HR and financial and these other things. And this is a big question to ask you. So, you know, I'm putting you on the spot, but that's why we're here. How do you see the future of work kind of evolving and taking shape in this new, you know, I call it the generative AI era for now. Who knows if next year, you know, a new breakthrough is going to give us a different paradigm.

21:14But, you know, you mentioned companies, your customers kind of looking for these efficiency gains, but reinvesting the money, reinvesting the time to do more creative, more strategic work, that kind of thing, which sounds like a best case scenario in a lot of ways. But I don't know, are there trends you're already seeing emerging beyond that, kind of indicating where, you know, work is headed in the AI age? Yeah, you know, I saw a quote recently that was saying, you know, you're not going to to lose your job to AI. You're going to lose your job as someone who knows how to use AI, right? And that's what we've seen, right?

21:46The people who have really taken the time to get good at incorporating ChatGPT or another LLM into their daily workflow are actually finding value and it's making them more efficient at what they're doing. And those people are definitely on the cutting edge, but a couple of years from now, probably everyone's going to be using LLMs in some capacity. And the sooner you kind of scale that learning curve, probably the better it's going to be for your career and your job opportunity. And so I definitely encourage people to play around with it and think about how you can just incorporate it in a day-to-day basis.

22:17Yeah. Yeah. I do a lot of writing and I find the models are great for kind of brainstorming first draft kind of stuff. And it's, yeah, it's become part of my flow. Right. Writer's block is almost a thing of the past. I did a first draft of like my, you know, my year-end reviews for my team. And, you know, I went back and edited them heavily, but usually when you sit down in front of that blank sheet of paper, it takes a little time to kind of get over that, right? And so you just, you just kind of skip that whole step. Yeah. Yeah. No, it's wild. Multimodal models are becoming more and more of a thing.

22:47And beyond the sort of obvious, this phrase obvious, but mind blowing kind of came to mind because it's, well, obvious you can, you know, images and audio and whatever, but it's also sort of mind blowing the applications. But from more of a, you know, again, pragmatic and the kind of work that you do and and what, you know, enterprise customers of yours are thinking about, from that standpoint, are there like new challenges, new concerns beyond, you know, well, it's multiple types of data, but are there new things that people are thinking about, worried about, excited about with multimodality?