The biggest AI threat still comes from humans | E31

18:14Kevin Mandia:Markets did not take this well. Softback dropped 13 % in Tokyo yesterday. Samsung and SK Hynix each fell more than 4 % in Seoul. I mean, I guess the big question is how seriously are we taking this?

18:26Soups Ranjan:Yeah. On the concentration point, though, Supes, I want to ask you about this because I think Kevin handled the, you know, is it autonomous swarms or is it people using AI that we need to worry about more the latter in his view? But concentration could, I don't know, conserve a lot of intelligence in these labs and away from startups like your own. So how does Sardine think about concentration in the AI game and defending itself against the possibility that these foundation labs slowly eat into your business domain? The world of fraud as well as security, it's really just a cat and mouse game.

18:56Soups Ranjan:So that's where I'll start. Right. And I, you know, I wanted to also tee off of, you know, what Kevin was saying. I actually found fully agree with his point that, you know, as we are at this new sort of, this is a new technology, whether it is open source models or closed source models or not, you know, this is actually an opportunity for us to uplevel our security game, right? The institutions who are actually going to be adopting red teaming tools or using AI to attack themselves or setting up bug bounty programs, which encourage them to do so, they're actually going to be far ahead of others in terms of like showing up their defenses.

19:39Soups Ranjan:So that's my first point. And then the second point is that, you know, even if you look at all the incidents that have been sort of talked about, like the hugging face exploit, et cetera, et cetera, it is all about like the instruction that was given by a human to the AI agent. In the case of hugging face exploit, you know, of course, there was a mistake. AI thought it's actually an assimilated exercise, except it was in an open internet. But then the question that we should be asking ourselves is, could we have given AI the right instruction so that it doesn't go out and do things autonomously, which it's not supposed to do?

20:16Soups Ranjan:And to me, that is a solvable problem. And then going to your point, the question you asked me about, you know, concentration risk of AI in the hands of, you know, a couple of frontier labs versus not. Actually, we see the converse right now. We see that the open-weight models are, in fact, getting equally good or coming pretty darn close to be as good as the closed-source models. I personally am running a local harness using Hermes, for example. And a lot of the tasks that I was actually relying on closed-source models, I'm now seeing that there are many of them that you can actually use open-weight models.

20:56Soups Ranjan:At the moment, I'm trying the American open weight models, but the Chinese ones are also, of course, they're ahead of the American open weights models and we will catch up. But the idea here is that I'm not afraid of, or I don't think that that concentration risk exists. Open weights models are actually going to pretty soon catch up. And there was another question of yours, but I'll shut up for now and then answer. It's interesting to hear you not be too worried about concentration risk. And the way I had this written in my notes was just, is GLM 5.3 enough? Does that get us enough security capability to be secure?

21:38Soups Ranjan:And it seems like in your view, these models on the open side are catching up quickly enough. But at the same time, we're seeing companies like Salesforce with their new COA model. And I think Sardine trained its own model as well, if memory serves. Yep, that's right. Right. Yeah. So we train our own foundational model to do fraud detection. I think our thesis is that, you know, the Frontier Labs, they are training their models on publicly available data. Right. And therefore, data is the new moat. Right. So companies like Sardine, we have over the last six and a half, seven years built up our own data consortium.

22:16Soups Ranjan:Right. So we service about four, 15 institutions globally, preventing fraud and financial crime. and we have like a consortium of, you know, 6 billion plus devices, 440 million plus consumer and 3 million businesses and trillions of dollars of payment activity that we've seen these consumers and businesses perform. So that is actually a proprietary data which Frontier Model Labs do not have access to because they're not in the business of fraud, right? And we can actually train our own models and that is what we are doing now to do better fraud detection. So we've traditionally used supervised machine learning or unsupervised approaches, which are traditional decision tree type models.

22:56Soups Ranjan:But just a couple of months ago, our AI team, we trained a foundational model for doing a card fraud detection. And that model performs, depending on the context, like either 40 to 65 % better. right so we uh we applied it to uh a couple of clients for whom we had never seen the data before and right off the box that consortium model was like either 40 percent 40 percent better for a consumer client and 60 percent better for a business client right the fundamental reason why uh in the world of fraud detection large language models or transformer-based models work is because there's a language of payments as well, right?

23:42Soups Ranjan:So for example, I shop and therefore swipe my card at a Safeway or, you know, a Starbucks, et cetera. And if somebody stole my card, they're going to transact with it at Gucci's. So the next token prediction, Supes being associated with Gucci's is actually highly unlikely, right? I think your t-shirt looks fantastic and you're the most fashionable person on the show today, Supes. Don't talk down to yourself. Kevin, I'm curious about how this applies to your end, because I can totally see how in the case of Sardine, lots of data, post-train a model or whatever they're doing, and then you can take something to market that's materially better.

24:18Soups Ranjan:On the other hand, when we think about cybersecurity and penetration of organizations, we think about usually the largest, most dangerous models, dangerous in scare quotes, mythos, fable, Astra, et cetera. This is a silly question, but I'm going to ask it anyways. Are you guys considering post-training a model to make it more dangerous to improve your ability to red team for companies? Or is that unethical?

24:42Kevin Mandia:No, you definitely do. Because that's where it's going. It's no different than people creating the virus of tomorrow today so you can get the vaccine ahead of time. I hate to say it. That's just smart. You want to be able to know your exploitable risk before the bad guys do. And I've seen hundreds of CEOs go through breaches, probably more than anybody else on the planet. and nobody deserves to go through those. And so we've built a ton of controls. You can't just, we hired 50 of some of the best Fortune 500 red teamers on the planet. In fact, when we went through their history, we had literally been hired to red team 99 % of the Fortune 100.

25:21Kevin Mandia:I'll leave one company that's never hired me out of the dialogue, but the other 99 did. Do you want to name which company that is? Because we're all so curious. No, I love their products. I use their products all the time.

25:31Soups Ranjan:Oh, it's Apple. Okay. It was Nike, but now they're gone.

25:34Kevin Mandia:Yeah. Yeah. Well, I don't think Nike is Nike Fortune 100. Yeah, but it's, you know, you got to, when every single thing our team does with humans, we have autonomous mode for our agent to run, semi-auto mode with like a safety agent running on top of it because it doesn't have judgment. You got to apply judgment somehow. We've created hundreds of thousands of evals. We've created classifiers. We run our agent when it goes on offense. And yes, we have different agents to scout and recon your network versus hunt and exploit your network. We have rules of engagement. We work out with our customers.

26:10Kevin Mandia:But our agent was trained with visibility into what all our operators have been doing this for 10 to 15 years do for a living. We know what the prompt should look like. We have a listener passively see every damn prompt going to the frontier labs when we choose to use them going outbound and inspects what comes back because we usually know what to expect coming back that's called you can put a bullet in the head of the agent if it does something you don't want it to do we've inject in and see every instruction it offers for months we've only been testing by the way production environments at 1a enterprises that's what we do for a living so we're running our agent externally in the wild.

26:51Kevin Mandia:But you've got to not just say, here's the behaviors we want it to have. I think you better really be thinking, and here's a bunch of behaviors we don't want it to have. And I do think there's a certain amount of domain expertise. If you're building an agent that does trading, you probably want a bunch of financial experts and investment bankers and folks that are aware of all the SEC guidelines and other things to do something. Same in the medical field, same in cybersecurity. If you're making agents tailored to do fraud soups, you're going to have fraud experts. You do need an expertise to get the behavior you want to leverage from AI that you get it.

27:29Kevin Mandia:And that's important. So there is a way to put guardrails on these things where everyone can benefit. But I think it's probably harder for the R &D at a frontier lab to pull that off.

27:39Soups Ranjan:Yes, I agree with that. Really curious now, hearing you describe the different agents you use and being out there in the world. What is your AI spend curve look like internally?

27:49Kevin Mandia:Well, I can tell you this. It used to take us five days and two people to look at complex web applications. Now it costs$5 and takes about 10 minutes. So everything in reality with that perspective is getting more affordable and more effective for everybody to get better results because of AI. This is a conversation we have on this show a lot where there's one idea that, Well, if you're doing high end professional work that has to be done right, like detecting financial fraud or cybersecurity, you need to be using the absolute best models on the market. And that's what you're going to be going with.

28:23Kevin Mandia:But now that we're having this conversation, it almost sounds like, you know, better, good enough. We'll get there. It's a or more of a mix. Is that what you are both finding that you can use a combination of the highest level frontier models and, you know, maybe slip in some Kimmy or some GLM in there? Supes? Supes, you want to go first?

28:42Soups Ranjan:Yeah, no, 100%. So I think there are certain tasks for which you don't need like a complex closed source model, which is more expensive in terms of token spend, right? And you can actually get away with just using an open source model. In fact, one of the theses I'm working on right now is, is there, is, could I create a local harness and a router where I'm routing between closed source models, which are running on my laptop. And then for complex tasks, I'm handing the task over to a closed source model. And that's why you've seen, right? Like OpenRouter got acquired by Stripe and there's like a whole crop of other without-taying companies who are popping up now, right?

29:25Soups Ranjan:I think there's a point to be made here that all CFOs everywhere are realizing that token spend has just blown up and everyone is not trying to optimize the token spend.

29:36Kevin Mandia:Yeah, and the way we do it, But I've got to trust it because it's a PhD from Stanford telling me. But we use every model and almost every assessment we do. And there was a time when we first started and people were like, I only use the American models and the closed models do the attack. I'm not convinced that's what you're going to be up against. We created a test, eval, 20 different kill chains, as we call it in cybersecurity. Get remote code execution here and how do you permeate to the crown jewels. And we have 20 different full paths in this. And we've tested Kimmy, you know, the GLM product, the DeepSeek product, some of the open labs and open AI and Proopic Gemini.

30:13Kevin Mandia:And we grade them on efficacy, speed and cost. And and by the way, we use Grok as well. And we looked at them all. And on different times, like one time, exploitable risk was found fastest by open AI. You know, three months later, it was something else. In general, what we saw was the time to get to exploitable risk was faster with the frontier labs that were closed. But here's the dirty secret for us. The last time we ran our tests across our 20 different full kill chains written by several people that have done this for over a decade, no lab found more than eight of them from point A all the way through.

30:50Kevin Mandia:All right. And this is something that if you put very experienced humans on, it would get the 20 out of 20 in a hell of a long time, but it would get there. And that doesn't mean AI is not as good as a human. Trust me, the AI, when we run it, we have found over 90 zero days at Armidin since January 1st. And some people would be like, that doesn't seem like a lot. That's a lot. Yeah. This is with zero source code review, zero source code review, and zero authenticated access. We're banging on a black box from the outside, guys. And that's bad when you find over 90 in a year. I don't have the exact number because there's 50 people in a room right here finding one every day.

31:29Kevin Mandia:But we don't find that without AI doing so much of the work that some of the unique people that can find exploitable code can find it. So if I had to peel back of the 90, how much was found by the frontier labs and AI, again, not source code, it's probably under 10%. But it's doing so much incredible work that the people that red team for a living finally get to have the fun stuff to do. Do I think that the Frontier Labs get to we've already taken out the beginner red teamer with AI? It's just you better get from beginner to intermediate really fast. You know, I think we're taking out the intermediate as we speak.

32:09Kevin Mandia:And for a while, you'll still need someone in the cockpit that's elite. And then you still need someone overwatching what AI does, I think, for some of the judgment charts. You better have a model that's literally observing what your agent is doing and the results it's getting. And every once in a while nudges up a judgment call because even humans dispute the judgment in some situations. So have the dispute with humans if you need to for now. So I do think a lot of the judgment we do will get percolated upwards to a human, especially in red teaming on production networks. But that's my long winded way of the PhD guy shows up every three months and shows me here's how all the labs are doing.

32:47Kevin Mandia:And here's the weird thing. You see the closed labs accelerate into exploitable risk. You look at all our evals, look to the right over time. And you're seeing Kimmy right there with them. Yeah. When it comes to what we do for a living, the open models are pretty much there. Not as fast, but way cheaper. So if you look at modern warfare, how do you want to do it? well cost is going to matter you know but again it comes down to the infamous if the frontier labs are five percent better do you want to pay 80 percent more maybe in some circumstances i think that's probably the right thing uh but so many tasks like soup she like you said boy the open models are delivering far better than human results very consistent and you can leverage them And I kind of like it to, you know, in the military terms, the open labs versus closed labs is kind of like we're going to deploy an aircraft carrier.

33:43Kevin Mandia:That's your closed lab show. And it's going to make its presence known versus we're going to deploy a bunch of PT boats and a couple, you know, 500 drones. Different efficacy, but you might have the same results over time. Interesting.

33:57Soups Ranjan:I thought you were in the Air Force. Why are you making reference to Navy hardware? I think this is a this was really cross branch here.

34:04Kevin Mandia:I did say aircraft carrier, but that's still maybe you're right. They're the ones. It's all the armed forces, U.S. are for they can they can mix their metaphors, I think, of the armed force. Here's I think when you got to go to managing between win and loss and not profit and loss when discussing technology. And that's why I love that I'm on with someone who's looking at fraud. That's a win and lose category. That's an important thing for us to prevent, you know, in the same thing with cybersecurity, where people are getting ripped off during this podcast. probably 50 people are going to lose 100 grand or more due to a fraud.

34:35Soups Ranjan:That's what I wanted to ask about because Supes mentioned that they're seeing a rise in deep fake based fraud. So we're seeing kind of like AI has improved. It allows for new things. People are using them to do bad. But when we talk about cybersecurity, Kevin, it feels a lot like we're talking about a couple of incidents from the major labs themselves and much less about picking a company at random. How's Kroger doing in cybersecurity terms?

34:57Kevin Mandia:So I love Kroger. Love those people. They're one of the 99 out of 100. They must be on the East Coast. I'm sure they're doing great. You know, hey, like I said, it's not that GPUs are on every single corner. If you are a criminal, you still got to get to, you got to get the processing power. You got to pick a model. You may have to jailbreak a model. We are not seeing the type of attacks that Armiten can do live. We're not seeing them in the wild just yet. That's interesting. So we're getting a respite based on compute and availability. And by the way, if you're a criminal, you kind of want to do a lot of these activities anonymously, which means either.

35:34Kevin Mandia:I mean, you got to pay for it somehow. So Bitcoin will be your preference. But if you got to use a frontier lab, you got to pay for it. You're going to probably just try to hijack somebody else's stuff. And then those labs do generally probably, you know, you're going to have to step over the ankle fence, as we call it, and get get the models to do things they may not want you to do.

35:53Soups Ranjan:It sounds like you're saying, though, Kevin, that the current GPU crunch, the current compute crunch is keeping us safe from nefarious use of AI models because all the computers being eaten by, let's just call them white hat companies.

36:06Kevin Mandia:You're not going to try to hack the whole darn planet as a criminal right now using AI. You're going to have to pick your targets and do some certain specific. There was a there was an article in Bloomberg last month. I just sent it around if somebody could pull it up here. Well, they actually like exactly says what Kevin is now explaining to us, that Chinese hackers are relying on DeepSeek because it's the cheapest. So like they're very budget constrained with what they can do. So they're doing as much as you could possibly do with the top DeepSeek model because it's so expensive. But obviously, when you have, you know, an American company like Armiden that's using a combination of all the most powerful models, they're not going to be able to sort of maybe keep step with that, I think is the idea.

36:45Kevin Mandia:A lot of what you do in cyber, by the way, you have very intentional and deliberate things that you do where you don't even need to tap the frontier labs. You don't need it. We've written over 150 custom tools to do certain things like identity security assessments and other things. And that's the difference. You're constantly going, do we need to go to a frontier lab? Yes or no. Which one? With the recon, we'll use more of the open models. But if we're looking for real exploit development, we'll probably, depending on time constraints or cost constraints, we would probably go to a closed lab.

37:14Kevin Mandia:you know, and get a little bit of a faster answer. Because we do see that arrow for finding exploitable apps faster right now in like an open AI or Anthropic or Gemini than DeepSeek. But again, it's still compressing. You know, there's going to come a time. It's not far-fetched. I think two years from now, it really doesn't freaking matter. When it comes to cyber, AI is doing, regardless of the model, it's taking cyber operations to a new level where you better or have AI on defense to defend.

37:41Soups Ranjan:Okay, Stoops, I want to make a slightly related point here. You were talking about training a model earlier. Kevin's talking about harnesses and testing suites and benchmarks and all this. It seems like the amount of intellectual weight you have to have inside of a technology company today to correctly take advantage of AI is incredibly high. One, am I reading that correctly? And then two, is that going to get easier or lower over time, therefore allowing more companies to experiment and build with AI in a way that does move the bar of technology forward as you're doing this, Ardine? Yes and no.

38:13We do see that in terms of AI startups,

38:20Soups Ranjan:there's a bunch of them who have popped up who essentially help you train your own models. So we ended up doing it ourselves. And of course, NVIDIA also, they take a very active stance in this. So they have a full team, which is just dedicated to helping companies like us understand how to actually go about training for initial models. So we took advantage of that. But then at the end of the day, there are those who recognize the threat from AI to their own business. And therefore, the ones who realize the threat, they're actually going to go and train their own models. and those who are just caught sleeping on the wheels, right?

39:05Soups Ranjan:But in terms of how do you go about doing it, it's actually still pretty difficult, right? The talent that you need for doing it is actually incredibly hard to find, especially when they are being scooped up by all the frontier labs for inordinate amounts of money, right? So that I think is the more difficult question over here. How do you go about finding the right talent? And one of the things we are actually doing is, you know, and by the time, I don't know when you plan to publish this, but by the time you publish this, we are actually establishing an AI lab within Sardine. We're calling it Sardine AI lab.

39:41Soups Ranjan:And we are going to be essentially working with academia. So we are announcing that we're going to have, you know, five grants,$75 ,000 each for academics, right? PhD students, master's students to come and actually work with us to build frontier models to advance fraud detection. Because we have the data, academics, they have the know-how. And so this is our attempt at sort of giving back to academia and also to address the question that I raised over here, which is it's actually incredibly hard to find the right talent to come and build the models. So that's That's one of the approaches we are taking here.

40:22Soups Ranjan:Is there a similar talent crunch, Kevin, in putting models to work in an offensive cybersecurity capacity? Because we mentioned how the GPU crunch is keeping us safe because there's not a compute to go around to misuse. But Sup's point about it being hard to access people makes me wonder if the malefactors are having a similar tricky time getting the right people to go forth and do bad things for them.

40:44Kevin Mandia:Well, I was going to go to the original question really, really fast, because when AI came mouth, any security professional that was leveraging it for whatever they were doing in cybersecurity, they were getting something totally new and different and awe-inspiring, no matter what. The problem in cybersecurity is if you are 99 % secure, that's terrible. I mean, because of the scale and scope of security. And so what I think you get from these models without that extra four to 5 % of serious experience and talent is probably a 90 something percent solution that feels great because nothing's ever existed like it.

41:20Kevin Mandia:You take any frontier lab and use its coding agent and you send your source code into it, it's gonna find a ton of vulnerabilities, a ton of false positives, but a ton of vulns. And that was never really available before. And so what's the response to it? Wow, look at that. I think most people will never know the difference between the true A plus in cybersecurity and a B, or hell, even a C. But AI is definitely bringing that C up to a B no matter what. And what we're working for at Armid is that last percent, because at the scale and scope of the Internet, I've never been able to explain this.

41:55Kevin Mandia:So I'm going to try it again and fail again. It's like talking about the size of the universe, 43 billion light years. No human gets it. But one fraud technique that works, Supes, that's better than all the others can make the same amount of money as a thousand other fraud techniques, you know. And that's what I'm saying. Only one successful type of attack pops the balloon on the Internet, you know. And so you got to get that extra four to 5%. 1A enterprises get that. The world really doesn't. On the talent side, so I don't know if that answered your question, but we're working hard to differentiate from the frontier labs.

42:29Kevin Mandia:But it brings the baseline so much higher than where it's ever been before. It's impressive. By the way, I use Co-Work virtually every day and I use Gemini every day. I'm still amazed by it every day. They'd like those be better than I know me. You know, so, you know, I get some people will be like, well, that's scary. But, you know, we're doing the most when you think about AI and how it's scary, it could be for biological weapons, could be for cyber weapons. We're building a cyber weapon. And I'm here to tell you, you can do it securely if you spend the time to do it. There's plenty of controls that you can put around software to anchor it and watch what it's doing, inspect what it's doing, train what it's doing, tell it what not to do.

43:11Kevin Mandia:and I think you can get to exceptional results and maybe you don't have to tap all the way to infinite on the models to get what you need. In regards to talent, we just said what I like what we did and I'm very impressed with my co-founders, David Slater and Travis Lanham, both Stanford folks, is we're right outside of Stanford's door. I'm pointing out here because we have about 50 engineers right out the door. We didn't do a distributed workforce. We're like, hey, if you're going to be an engineer at Armadin, you're moving to Palo Alto. And then we broke a bunch of labor laws, probably grabbing Airbnbs for everybody and paying for them to move here.

43:48Kevin Mandia:But there is a different energy in the room when you're not coordinating across geographies, not coordinating different time zones. You're just in the same damn room. You got a question, you can stand up and ask it. And I think in an AI shift change, like the world's never seen proximity freaking matters. So I think that helps us get talent. That's why we kind of like the Palantir playbook. We're going to set up right outside of Stanford and try to hire everyone. As they come through the gates on Palm Drive, we just tackle them, throw them in a white van and drive them to the office. It's not quite that quick, but great talent at MIT, great talent in the colleges.

44:24Kevin Mandia:And they're born AI native now. They're coming out of college doing prompts to figure out how to get a better prompt. I mean, it's just different mindset than folks that clacked away at keyboards like I did with my CS degree.

44:37Soups Ranjan:Supes, you've been nodding your head. Do you want to jump in here? Yeah, no. I was just thinking a lot. I'm in Berkeley. Our headquarters is in Berkeley, and we're trying to do the same thing with all the Berkeley students. We were founded during COVID, peak COVID, and we've actually ended up being a completely sort of a remote first company. But now, just like earlier this year, we got an office in Berkeley and trying to hire more folks local as well. For companies like us, we cannot outbid an AI engineer who's going to get like millions of dollars of salary. So we've had to sort of, you know, find talent elsewhere in the US or, you know, abroad as well.

45:15And that has helped us quite a bit as well in the following way that starting via the remote first company, we are also a global company.

45:22Soups Ranjan:We have, you know, often customers in a country and we have, you know, an engineer in the same country who doubles up as a salesperson as well. or as well as a forward deployed engineer is helping us sell in the country as well.

45:35Kevin Mandia:When you're hiring people to help you build a specialized frontier or foundation model, like what are you, like lock me through a little bit of the hiring process. Like how are you evaluating, is this the right person who's going to help me train this model that doesn't exist, that only exists in my head?

45:51Soups Ranjan:For us, it is the talent that you look for, for training AI models is not any different than what you look for, for a machine learning engineer, right? So we've really just given that sort of task to our data science and machine learning team. There is a separate skill set needed to put those machine learning models in production so that at the time of inference, you can use those models in real time. In our case, the problem is actually even more sort of challenging than your Frontier Lab models, which are essentially all about languages or video or images. in our case we're talking like card transactions right so you're talking like you know um uh kevin is swiping his card at a starbucks and within hundreds of milliseconds we have to be able to apply the machine learning model you know uh utilizing all the historical transactions that kevin has ever done doing a matrix multiplication between his vector of transactions and this large language model which we've built which could be like hundreds of millions hundreds of millions of parameters long, right?

46:57Soups Ranjan:So doing that in real time within hundreds of milliseconds is actually a very fascinating problem which we are encountering right now. So to make the long story short, the talent that you need for both training a model or deploying it in production is nothing different than what we've traditionally done with machine learning. It's just you have to just dig deeper and find the people who are really good at it. That's all. So how much money have you guys raised total at Sardine? We've raised... Let me do the math. I always forget. Take your time. It's pretty complicated. Upwards of 150 million. Okay.

47:38Soups Ranjan:I'm just... I'm in awe that we're in a moment in the era of technology when you can raise$150 million, be in business for six, seven years, build a substantial revenue base, be worth quite a lot of money, and still say, you know, we can't afford to compete for this cohort of talent. I don't think that was the case when everyone was worried about Google hiring the entire crop of Stanford engineers, or maybe it was Facebook. But that seems like a really material disadvantage. Is there any way to combat that as you look to staff up Sardin as it grows? Or is it just like, wait for the fever to break or those two labs to go public?

48:13Soups Ranjan:Yeah, fair question. That is where the playbook that I described is what we're adopting, which is right, like we're starting or starting AI labs, we're going to go and essentially hire more researchers. And essentially, these are think of them as interns that you then convert to full time hire after they graduate, right? So you got to be trying to go earlier.

48:37Kevin Mandia:Yeah, for me, it's, you know, I'm a self funded startup guy, my first company, it was founded in 2004 with no funding. So and it was a cybersecurity company, and I went seven years before doing funding. So how the hell do you compete when you don't have money? And the only way to do it is mission. And I know that sounds Boy Scout-y, but that's the damn reality. You've got to literally say, we're going to secure the world, end the story, and think big and go for it. We're going to create a layer of defense where everybody can use at 1A Enterprise and maybe even in your personal lives, confidently operate on the internet without fear.

49:12Kevin Mandia:That is a vision that you want to have and you can attract great talent. Again, you still have to make it financially rewarding too, though. And that's the nice thing about being a startup, right? Supes, I mean, we can create great value, but man, there is no way to compete on dollars alone in Silicon Valley if you're a startup. I actually believe that. It's got to be believe in the vision, believe in the mission, and you want to move fast. You don't want to be comfortable. Some people call it wartime versus peace time we're in wartime at a startup like it's you know you got to feel like you're hacking away you're hacking out of the corner every day you're like a you know period you're a starving actor every day you show up and go what's my next gig how do i get it done and i think that's just a special thing but when you find it um that it's it's a wonderful merger of what people want to do and what the world needs and that that's that's what you look for Mission is good.

50:12Soups Ranjan:But Kevin, I'm curious. Mission is good. And I'm glad that that's working for the company to get the people that you need. Does your personal celebrity, and I know that's a bit of a wrong word here, but you're well-known in the industry, major exit, Amazon's board, et cetera. Does that change how hard it is for you to get the people that you want when you're not competing on just gross cash with Anthropic, for example?

50:34Kevin Mandia:It has to, right? I mean, people probably wanted to play with Tom Brady. I don't want to liken myself to Tom Brady, though, but you'd get on teams where they learn. Like the best thing I've ever felt is when I sit around the table, no matter how the company's doing, if you can sit around the table and just look around and go, holy crap, these people are great at what they do. You're going to stick around. You just are. You know, the company does have to win as well, though. But you don't just have an exit because you were stinky at something. You know, you got to be pretty darn good at it. And I think people are attracted to talent.

51:07Kevin Mandia:talent will attract talent. But every time we interview folks, though, we do go at the mission, you got to love cybersecurity. And I've always been amazed. Like, I think suits your mission is so important. I'd like to be doing this with you, because fraud's terrible thing to go through. And to prevent that matters. I wouldn't like we need a third bubble with the person preventing cancer. Let's get them on. You know, because it's those are the kind of founders that I think the mission matters most. And like every day I wake up, it's not the financial results I'm looking at. And I know that sounds like BS, but it's not.

51:41Kevin Mandia:It's are we the best in the world of what we do? And you know it when you're not. I can tell you that. And you got to keep working till you are. And I think that can attract great talent. But I think I've gotten and it is easier with the raises you can do today. I do believe that. I think it is easier, though. I haven't done the self-funded startup, I cringe when I think about it now, you know, uh, I wouldn't do that again. And, uh, and I think we got a great moment in time where the startup is necessary, you know, with this amount of change in the world, um, you need speed and, and there's no doubt the littles are a lot faster than the big for sure.

52:21Soups Ranjan:But you know, with your first company self-funded for a while, this company, you raised, I think about 190 million between your seed and your

52:27Kevin Mandia:series a who's counting who's counting me reporter sorry uh i have spreadsheets in my dreams well you

52:34Soups Ranjan:have to um yeah when you're up when i started covering venture capital that was an entire series a fund you know and so the the amount of money here is getting to be a little bit uh not terrifying per se but almost concerning so people talk a lot about these large seed runs for example kevin and you raised like i said a pretty big pair ground um what are the expectations like in terms of near-term performance for your company going from starting to having nearly 200 million raised?

53:02Kevin Mandia:Yeah. I want to be in the top 1 % in every business metric as we go to this thing. So everybody goes right to Wiz and how fast did he get to 100 million in ARR? We want to beat. We couldn't have better market demand, guys. I mean, it's incredible. Everybody wants to know, can we withstand an attack from the money? We do it. We're done in five hours. Like, hire us today, we're done today. Yeah, we'll launch an attack like you've never seen and you'll get your answers. Yeah, I like that expression. No, I mean, you're killing me. I know this is not billable hours, but I would take at least a week, just make him sweat a little bit, make it seem like I'm doing something.

53:38Kevin Mandia:Well, the what to do about takes us a lot more time than the attack, right? And so we hired some of the best remediation folks to help us do that. You can't just find problems. I can't just show up and say, hey, you got nine darts sticking in your back. And then I don't just pick them out. You know, you got to help out and you got to show what people have to do about it. But you're right about the dollars going in. The minute everybody's raising money, it's kind of like you need to show up to to do that. And it is a different world today from my startup days where I could take seven years where I did around and we had to execute profitably.

54:13Kevin Mandia:Now it's, hey, if you're not 100 million in ARR in like five months, what are you? You know, the speed and compression of everything. I think it's good and bad because I think it's good for the winners, but it's really hard for the losers because you can raise more money and survive longer. But if you're not in that best in the world of what you do, I mean, the value starts to taper off pretty quickly. So you got to aim for, you know. Yeah.

54:36Soups Ranjan:But Supes, you know, if you think back to when you started Sardine, you know, seven years ago, definitely a different era. We're talking about, oh, my gosh, I was pre-COVID. Well, just about COVID. So that's, you know, the SaaS boom and so forth. you didn't have quite the same growth pressures on the company when it was very nascent. Did that help or hurt Sardine's overall growth trajectory, do you think? We are in a very different business, right? So fraud detection has traditionally been sort of done by incumbents who are usually a subdivision within, let's say, the large players, right? Like NASDAQ owns a company which is competitive to us, or LexisNexis owns various companies which are competitive to us, but you wouldn't really find large independent companies in this space.

55:23Soups Ranjan:So for us, it was actually a different sort of skepticism that we had to overcome in the minds of investors, which was that they would always look at us and say, hey, there's no large player in fraud detection. And our answer, in fact, and Kevin would probably, I'd love to get Kevin's reaction. Our reaction to those questions used to be, hey, look, just like this did this did in cyber security which is they built a platform right we are doing exactly the same thing in fraud the reason you don't see a large sort of company in this space is because there's lots of point solutions and sardine is a platform for handling all your fraud bsa aml right which is like all your kyc id verification and aml anti-money laundering transaction monitoring all of that in one platform underneath the hood there's like 20 products and it took a while for like everyone to sort of come around that vision and now we see like several other folks who are sort of copying the vision and trying the same approach but now the good news though is that you know we've been growing faster than ever so the last couple of years we grew 100 % year over year right and you know we've stayed largely close to that vision.

56:37Soups Ranjan:So yes, in a lot of ways, you know, we've never sort of, you know, compared to your newer type of AI companies. And also, you know, the buyer in our case is actually very different. In our case, the buyer is like a head of fraud or head of financial crime compliance, right? So Bank Secrecy Act AML officer. And I don't know if you know this, but the BSA AML officers are personally liable for any you know uh uh any aml issues right so they can actually personally liable not the company that individual that individual they're personally liable that job sucks yeah so uh so therefore you know they they and also like uh there's a lot of regulatory uh sort of uh uh things that they have to work around so the buyers of our technology like the heads of financial crime or heads or BSA ML officers, they're usually the more skeptical ones.

57:34Soups Ranjan:They don't necessarily want to immediately adopt AI. So it takes, you know, it takes them, it's taken us like some time to sort of convince them that, hey, AI can actually really help you. And I'll give you an example. There's a bank, you know, so first and foremost, right? Like in a lot of banks, about a good 30 % of the workforce is dedicated to just, you know, routine works, right? Like onboarding queue reviews or, you know, sanctions reviews or transaction monitoring reviews. These are nothing but, you know, like you're looking at, you know, this transaction, this wire transfer that was made by this person to that person.

58:14Soups Ranjan:Should they really be doing business with each other? You're drilling down into it. Or for sanctions review, you're looking at this person's name got flagged in some sanctions hit list. Is this really that same person or not. And it's taken us some time, but we've actually finally managed to convince a lot of banks that you can actually use AI to automate a lot of these things. And for one bank in particular, we automated like a good 90 % of the sanctions reviews. That's 90 % without missing a single true hit. A true hit would be like you're missing an Osama bin Laden and letting them on both right so that that never happened but unlike unlike ai for support where you can go go live immediately for us to go live with this was a six-month journey yeah right because the the accuracy level required here is 100 percent yeah no i'm not personally liable yeah because they're personally liable right yeah and the and the way we went live about it was that we said okay first we will backtest all your sanction sets against our AI agent.

59:20Soups Ranjan:And we found that every time it was doing 100 % accuracy for identifying true hits, right? Then we actually turned it live in production, but with a human in the loop. So human was reviewing what AI decision 100 % of the time. And over a few months, we tapered it down to 10%. And every single time we did that, there was an eval running, which made sure that there was 100 % accuracy in identifying the true hits. Of course, there was some inaccuracy. AI was actually more sort of pessimistic. So it did identify some people as a hit, which human had to go and then say, oh, no, no, no, this was not a true hit, right?

59:58Soups Ranjan:But then at least we tuned AI to fail in the right direction. Right.

1:00:03Kevin Mandia:More cautious rather than less. I'm curious, one quick question for you, Supes. So I'm assuming that your financial data that you're sucking in to train these models has got to be anonymized in some ways. That's what we always hear. All the data is getting scraped. So it's not personally identifiable. But isn't that when you're looking for financial fraud, isn't the personally identifiable stuff part of what you're looking for and training for? So how does that work? How do you remove enough data so that it can't identify me, Lon Harris and my purchases, but it's still going to recognize a fraudulent Lon Harris down the road?

1:00:37Soups Ranjan:No, in fact, that's not the case because we comply with all the various data privacy regulations. We comply with GDPR in Europe and CCP in California. We have SOC 2 Type 2. So the way we approached fraud prevention is that whenever banks or fintechs or payment companies, they share data with us, they are opting in into a consortium. And they are, in fact, yes, they are sharing the PII information with us. But with the express, they review our security posture, et cetera, and they share it with us. And the other thing that we do is that, you know, when we build a consortium model, we do anonymize it at that time.

1:01:27Right.

1:01:28Soups Ranjan:But then, you know, like for certain things, you have to rely on non-anonymity. Like you have, for example, I'll give you another example, transaction monitoring, right? Like if I have onvoted a business, which is a restaurant, I want to monitor all the transactions, all the wire transfers, ACH they're doing to the counterparties. traditional AML approaches they only looked at transaction amount and velocity and that is why there were lots of hits but what Sardine does is we go take a look at does this counterparty have any reason to be doing business with this restaurant if the restaurant made a hundred thousand dollar payment to you know an auto parts supplier or some dual goods merchant right then you start questioning it right but then it's important to know the nature of the business of of your customer, which is a restaurant and of that of the counterparty.

1:02:20Soups Ranjan:And that is where our AI agents come in. Also, like we have AI agents which go and just given the name of a business, it'll go on the broader internet and find out and classify the nature of the business and then come back and see if this transaction makes sense. Got it. I want to go back, Supes. You said that banks have a lot of staff that are just checking over transactions and starts and you took the number, the human input from like 90 % to 10%. What was that metric again? Yes. So we automated 90 % of it. So as in we have still a human in the loop, 10 % of the time reviewing what AI did and it's auto-decisioning.

1:02:55Soups Ranjan:But we cut it down for 100 % human labor with just 10 % human labor. Yeah. And then Kevin earlier said that, I forget the exact phrase, but something like the junior red teamer is defunct now and maybe even the intermediate one, Kevin. That's a fair quote?

1:03:09Kevin Mandia:I think if you're a beginner, you'll have, AI is going to get you to intermediate really quick. You're going to learn a lot faster. But yeah, I think when you look at the performance of the Frontier Labs and the open models, it's better than what I used to be when I started my career.

1:03:25Soups Ranjan:Well, there was a lot of fear a year ago about white-collar jobs are going to go away. So far, not the case. Unemployment is still pretty low. And it's become almost a faux pas to discuss the possibility of automation predicated on AI taking away some jobs. But whenever I talk to founders like you two, who are serious and are building real companies and are applying AI in the real world, I just keep picking up these signals that there's chunks of labor that we're going to do away with. So, Kevin, do you think we should be thinking about that at kind of the societal level? Or is this just going to be normal churn in the business landscape as new technology is adopted?

1:04:03Soups Ranjan:I'm trying to figure out kind of like the pace of that change, if it makes sense.

1:04:07Kevin Mandia:Well, first off, you'd rather be a builder at a time where you're seeing a shift change like this and somebody who's been around and I've got 10 ,000 engineers and how's it going to impact them? Because then it becomes a daily problem if you're uncertain and you have 10 ,000 people. With us, we're just starting to build up so many different functions, like our go-to-market. And so we're very mindful, how can we leverage emerging tech to do the job better than humans? And what I'm finding is kind of need people in the NFL cities to sell in the NFL cities. You know, so the sales force, I don't know what the impact is other than the training is probably going to get better.

1:04:41Kevin Mandia:They're going to be more empowered faster and they probably have their own AI tooling to make them more productive. But, you know, you're looking at every discipline and how does AI impact it? It impacts content for marketing. No question about it. It will impact analyst roles. Like I built a huge Intel analyst division at my prior company, hundreds of people. and in hindsight you know looking at some of the things we're learning now oh it's going to be a lot less people but everybody every ceo of every company is trying to figure out what is the manpower impact in any direction of ai over the next few years and i'm not convinced any of them has a definitive answer thank you there yeah you know i really don't think they know and we're all talking to each other going well like i even i asked how did your go-to-market change because of ai The answer is, you still need AEs meeting people and people are still buying stuff from people.

1:05:33Kevin Mandia:Steak dinners are never going away in enterprise sales. Yeah. In fact, actually, enterprise sales even matters more now because you have to grow faster than before. But at the same time frame, you can't, you know, you've got to have a guy in Minneapolis. You've got to have a guy down in Florida. It's everywhere. So I'm looking at it and studying it. And AI is making every human better at what they do. We won't hire as many red teamers. You know, because we're a product company. We got about 50 of them. We'll hire them when we need them. But I actually look at them now as they're R &D. That's what they actually really are.

1:06:07Kevin Mandia:You know, they're creating the evals for the product team. They're not really red teaming our customers. So I'll stop there. I'd love to hear Supes experience. I just think if you're at a big company, you really are thinking, well, if we're going to hire anybody new, show me I can't do it, you know, first, you know.

1:06:24Soups Ranjan:Actually, I'll reframe the question in the following way, right? Like, despite all the advances in technology, right, the amount of money laundering that still goes on in the financial ecosystem is still humongous. That number alone is just like bonkers compared to the reduction of, you know, human workforce AI can achieve. So my, you know, whenever someone in our space asks us this question, are you taking our jobs away? whenever a BSA, ML officer or fraud ops asks us. My answer to them is that no, actually we are enabling you to do your job much, much better because look at all the fines that banks are facing.

1:07:02Soups Ranjan:Look at all the money laundering that doesn't get detected. You can actually perhaps catch all of it. And then the real benefit or the efficiency will come from actually making payments cheaper because that interchange fee that we are paying to Visa and MasterCard, it actually is for fraud detection, right? So that is the net effect you will have. I don't think we're going to see in our space that folks are cutting down fraud or compliance ops teams. They would, perhaps to Kevin's point, the junior jobs go away, but the folks who start to use technologies like ours, they actually get better at the job.

1:07:38Soups Ranjan:And you start seeing, hopefully, you start seeing net reduction in payment fees as a result. See, everyone listening to this podcast, you can have a conversation about employment in AI and AI without everyone shouting at one another and claiming that everyone else is trying to execute regulatory capture. All right, before I let everyone go, Supes, Kevin, I want to hear what you guys are talking about other founders on the famous group chats that people in the audience may not be aware of. So, Supes, just staying with you for now. What are other founders in your area and sector talking about worries, concerns, things they're looking forward to?

1:08:12Soups Ranjan:Kevin, do you want to go first? What do you think?

1:08:16Kevin Mandia:Yeah, you can go first. Always worried about the market shifting, you know, aliens land, World War III starts, whatever. Some non-controllable event that freezes everything, right? Always worried about that. Second, you're always worried about the noise, especially in cybersecurity. There's only like five ideas and 7 ,000 companies doing them. And if you're really good, it is always frustrating. Or if you perceive yourself as good. It's always frustrating knowing a lot of these places have become marketplaces where people are showing up that don't have 35 years of doing this. It's not in their DNA.

1:08:49Kevin Mandia:What's in their DNA is, geez, I've always wanted to be a founder. I went to Harvard or Stanford and I'm going to lick this cybersecurity problem. Really? We've been doing it for 25, 30 years. We're still, you know, it's, you know, we're working on it. But I hate the noise. And I would hate to be a buyer trying to figure out all everybody saying the same thing. You know, I think founders are always worried about that. And then one thing I would ask, you know, when we're talking about AI and how it's changing the workforce, one thing that changed after my 30 something years of working is I didn't even know this, but I have a very young company, mid 20s, late 20s.

1:09:25Kevin Mandia:they record every damn meeting we do and have an AI summary of every darn meeting. And it doesn't mean we hire less people. It actually may even augment. It gives us this tremendous focus on what really matters. I used to spend my time as a CEO going, what the hell should we focus on? Now I have AI just telling me every day and it's right. Here's what you need to focus on. And once you need to focus on stuff, you can galvanize the team around it. So information, alignment's never been easier. I hate to say it. and AI is allowing us to get in alignment. And then you just put all those troops into that purpose.

1:10:01Kevin Mandia:You're not scattered. And I don't know if that's a resource diminisher up or down. You just move faster and more effectively.

1:10:09Soups Ranjan:All right, Supes, you've had a minute. Follow up, Kevin, see if you can talk about it.

1:10:12Kevin Mandia:I was even stalling at the end there, but I don't know, Supes, are you employees recording everything you say and getting an AI announcement? Actually, before we go on, they're recording all the conference rooms because you're in person. When I do presentations and everything, you always get the ad summary. When I'm doing meetings with people, it's become the norm, right? That's a problem that didn't exist. In my day and age, you'd be like, hey, man, I don't want your – my instinct was always, don't record me. I don't want to be recorded. I don't want AI mulling about what I'm doing.

1:10:41Soups Ranjan:Supe sent his recorder to this recording. I mean, that's how much he doesn't trust us.

1:10:44Kevin Mandia:We're being recorded right now.

1:10:45Soups Ranjan:Our recorder is in here too, yeah, 100%. No, we – yeah, because, Kevin, we started doing the COVID days and we would vote first and we were doing selling remotely. So we actually have a culture of recording every single meeting as well. And what has happened then is that we've now, like when we onboard a new salesperson or new customer success person, we just point them to, you know, the best sales calls that we've had, right? So that becomes a part of the training. And also like after every meeting, you know, a lot of these recorders, they actually will give you like a summary email that you can send with all the action items.

1:11:20Soups Ranjan:So that makes our jobs for follow-ups a lot easier. Going to the original question here, what are other CEOs, what are we chatting about in our sort of WhatsApp groups or what have you? I think one of the themes that I'm seeing is like everyone is trying to make them, all the CEOs are trying to make their jobs more efficient as well, right? So everyone that I know of is trying to like, you know, the CEOs who are hands-on with AI, they're trying to build their own harnesses or they're trying to automate a lot of the work that they have to do. Kind of like to Kevin's point, what should I be focused on today?

1:11:57Soups Ranjan:AI is really good at helping you uncover everything going on in the company. One of the problems I have right now, which I'm personally actually trying to build an AI to help me with, is I'm a very hands-on founder and I like to get involved in all the weeds. But now that we've scaled to 450 plus customers, there's 450 Slack channels because every single customer has a Slack channel. And in the early days, I used to read every single Slack channel, but I can no longer do that. So I'm literally just writing an AI agent, which can go summarize every single channel for me and tell me which one should I be paying attention to so I can hold my own teams accountable.

1:12:36Soups Ranjan:Well, guys, I guess there's no more privacy at work. Humor is off the table and you better not spend too much time at the LaCroix fountain. Then someone else will check your productivity stats. Lon, thanks for having me back, by the way. Thanks for coming back.

1:12:48Kevin Mandia:Yeah, what a treat. Jason could not make it. We were so glad to have you step in. And Supes and Kevin, thank you so much for being here. Amazing job. Great to have you. Just a quick reminder for all of you watching, if you want to check out our sub stack and keep up, we are posting new demos. We are posting new videos every single day. Go to thisweekina.ai to take a look at that. We'll also have that in the description. Thanks for joining us, everybody. We'll see you next time. Bye-bye.