Can we defend against AI hacks? | Dozie Anazia

0:00You're typing on your computer. The AI is able to know what keys you're hitting. The pitch from where the location of the key is on your keyboard can give away your password. A finance employee joined a video call with people who looked and sounded like the company's CFO and colleagues and then transferred$25 million-ish.

0:18Parth Patil:You should now, like, this is now the world we live in where you can't really trust real-time video and someone can put your face on, someone can wear your voice, kind of like a mask. The token grantee program gives$1 ,000 a week in tokens to high potential creators already deep in AI across film, gaming, comics, print, and digital art. There are no tool restrictions. The freedom to choose is the point. Grantees also get access to my own custom fleet of agents and an ongoing collaboration with me. The goal? To close the gap between an idea and the world. Part of this tokens program is showing how AI is, you know, kind of revolutionizing creativity, building, you know, and kind of having a variety of folks who are using tokens, tokens to the future in order to kind of build and to show what they build and illustrate this for other folks and kind of help see the future.

1:20And so I'm doing this with Parth Patil and Parth. I'll hand it over to you to kick off the episode.

1:27Parth Patil:Awesome. Thanks, Farid. So super excited to have you here today. Today we have a really good friend of mine. We have Dozi Anasia. And we're really excited to have you on the Token Grantee Program, especially because Dozi comes from a slightly different background from everyone else in the program. Dozie's background is in cybersecurity. But we'll get to that. I want to go back to how we met. I think it was five years ago now. I met Dozie on Clubhouse five years ago. Yeah, back in 2021. We were hanging out on rooms on Clubhouse, talking about games, talking about science fiction, nerd culture, and then just where the future was headed.

2:11Parth Patil:So why don't we take people back? um you know introduce yourself maybe like where we were when we met and then where we where we are today and what you're up to yeah um that's I think that's that's a pretty good description we just randomly met on clubhouse as as one does um it's 2021 and I don't actually remember exactly where we met we I think we just had like the same group of friends and um we're talking about like software development ai data like all the like tech stuff and um i think we just kind of like serendipitously like found each other because we both like would use gaming as like a metaphor for a metaphor or philosophy for just everything yeah so i always found i always found you interesting because you did that and i also did the same so just describing life in gaming gamify life was like kind of like absolutely how we started like having conversations with each other but yeah yeah and i think especially as uh ai came online in 20 cat gpt in 2022 and then gpt4 in 2023 we were we were pretty early like there's a huge early adopter community on clubhouse as you know and we were early to playing with these tools and seeing what code generation was and i was kind of like oh my god we need to unpack this together right like i'm seeing what it does to data analysis Dozie's in school studying computer science.

3:45Parth Patil:So clearly both of our fields are rapidly being transformed. And yeah, it also, it's like when you find these people, these teammates, and you start exploring the unknown, it's so much better than doing it alone. And especially as agents and AI gets more powerful, I think it's really, I find it really helpful to have someone like Dozie around having, you know, forcing me to reconsider how many permissions I'm giving my agents. So yeah, I was just going to say that it's funny you brought that up too, because when I was in school, I think it was like my second to last semester, my professor, I took an artificial intelligence class and that was like 2021.

4:30And this was like, he was showcasing open AI and open AI was basically giving you access to like Atari games and then you you could just um basically put it in your a Google uh collab notebook or uh or an Anaconda notebook and you just play like Pong or something like that in like in a in a in a web interface so yeah like I was on OpenAI like 2021 2020 so that was yeah that was my first rendition of it so by the way I totally get how you guys are like besties I mean It's like all the keywords that I associate with Parth. It's like gaming, clubhouse, AI. It's like check, check, check, check. That's totally fucking awesome.

5:22I'm curious about how this also kind of connects to your current day with AI. There's a whole breadth of projects you've worked on with tokens, gaming experiments, you know star collector letterboxd dupe um all the way to you know kind of like part of it is um you know we're not going to go into we're going to stay fairly general and externally known examples but also you do a bunch of stuff in security and you have a security expertise and so i'm curious about like kind of what your your your engagement with ai has been across this range cover some of them and then interrelate them some yeah so it's it's interesting because i so i'm working on the gaming projects the gaming projects are are fun because it's just like i'm just my own play tester for the game and then whatever i don't like i i prompt it again and and then i'm trying to figure out like how to how to get to like that that x factor of playability um and then just i my brain is just all over the place adhd uh i can't focus on anything to save my life but i have so many ideas and um specifically all of it does encompass um security because if you're going to put any of these things on a website you have to do some sort of like sanitization of the code and and ai does that for you uh codex does that and the crazy thing right now is that the the last project that i've been working on so so my my recent project is called um it's called breakfast and it's a i'm making a mobile breaking news aggregator okay and And so when I finished the project, I do it in Claude.

7:23And then I moved it to Codex to do the code review. Codex is like, this is very vulnerable. There's a cross-scripting vulnerability in it. So the last couple of days, I've just been trying to code it properly. And I was building a platform that just does like a code review of the URL. But like the AI is kind of already doing it. But like if you don't have the AI, then you can just – with the other platform I'm building, it's called Aegis. You can just throw in your URL and it just like – it kind of just scans it and tells you like if there's any port vulnerabilities. It does like all the OWASP standards.

8:12So yeah, it's just security encompasses everything. Like you're never really going to get away from it. And yeah, it's really important, especially right now, especially the way AI is going. Well, we'll come back to some of the interrelationship between the gaming and the cultural stuff, because it's the breadth that's also one thing, but since we're on security, what scenario do you worry that many security folks, especially ones who are maybe not as deeply versed in AI yet, of scenarios that those security people are still treating as science fiction and that they should be thinking, actually, the future is already here.

8:55It may be unevenly distributed. so that's funny that you use the the use it science fiction because i feel like we're we're living in the science fiction as we speak we're this we're in real-time science fiction because there are so many different attack surfaces now and um for for example i saw something with uh led light like there's a there's a way to there's a if you have like an led on your like your keyboard or something like that there's a way for you to like view what's happening from another surface from another machine through the led light which is crazy or like if you're if you're typing on your computer the the ai is able to know what keys you're you're hitting which is insane to me because you like if you can hear someone typing their password then you have their password which is like unfathomable to me how the ai can do that it's like if you were like in the 90s if you you would like mask like the the the phone sounds from your right from when you're typing right so but now it's just like literally the pitch from where the location of the key is on your keyboard can give away your password that's insane to me

10:21Parth Patil:that is crazy um no i feel that i feel the same way i mean we play you know my favorite game is cyberpunk and i look around in cyberpunk i'm like wait a minute almost every single thing in this game is definitely doable in the real world um where you know just just like the programs that we can talk to that can do a lot of the things that were previously manual and then thinking like oh you can clone them okay so now we have scale we have like parallelization which is both a huge benefit and and a negative right so i can actually have a bunch of these these agents looking at what i'm working on and adversarily kind of like critiquing the things that i'm making um yes it definitely feels like we're we're living in a science fiction world yeah i mean and now it's now it's it's going to be like autonomous like pen testing if anything um because because we like we've i think we were having this conversation before I don't know what's going to happen with like just the defense aspect of it because now it's like now companies are like employing like third party companies to like do their pen testing.

11:28But now you now you're just going to get an agent to do it for you.

11:31Parth Patil:Right. And then like you really just need to get like a third party company to audit the agent. but at the same time like now now you have like malware that's undetectable it's just like endless possibilities of what can happen um like well actually it's not it's not endless right because if you're the one that's making the software you have like an information advantage right so you know everything you have like the whole code base so you know what the possible surface area is where an attacker has to find the opening right and then and but then you have like Like the types, the, I think the incentivization of, of this type of thing would just be to like, if you're making that software, you, you just create a backdoor for it.

12:20That's, I think, and that's like where the, I think that's where the like disconnect comes from when, when people are making these types of things, they're, they're doing it for the money. And if, if they make a backdoor for it, that's, I mean, that's where it's going to go. that's really what happens when you're we're trying to find ways to like break into some of these these these uh uh into the software the backdoor is already there it's just you have you don't have the knowledge yet you know uh right right this is something i noticed

12:53Parth Patil:about all my friends in security it's like you kind of like benefit from this like assume the worst is already like right possible assume that like assume it's not secure and that we're only just like you know actually it's probably the best mindset in security is that assume you know understand the blast radius and assume that it's not secure and then start looking for everything that you could that could possibly go wrong and contain the the uh the possible um you know blast radius and then that's right the attacker the attacker um you know the attacker only needs to find one opening and then once you have the speed of ai you can kind of move they can move very quickly um right right and and that's that's honestly the scariest part is the fact that like if you like for example if you if if the attacker already has like your information um and like they know what like bank you you bank with they can they can clone like if you have ai you can clone like the the web page that the login web page to your bank and the the velocity in which you can do that now is like it's frightening so that's that's you can just like deep deep fake a page or you know vibe code but you can make a website yeah it's a website you can vibe code an attack essentially yeah but i think one of the things we should we should linger on this a little bit because the question is a changing services of offense advantage descents advantage you know there's a question of can you you find only one thing you create credentials you can be able to do that you only have to find one hole whether it's a you know clone bank or anything else but um you know say a little bit about kind of like in this kind of offense and defense side you know there's there's both the ai making individual attacks more sophisticated but there's also like many many different attacks and making it a lot cheaper to run yep those are all kind of things on the offense side and go through some and then also like you know uh what are the things that we need to be doing on defense given this too well well that's the thing like that's that's the thing i'm not sure about because when it comes to blue team like you if you're so if you're if you're an attacker um you want to do it like in a black box like you don't give the attacker any information and that's that's how they like they have to figure out their way into the into the space if they can do it then then okay then the blue team does the research based on the uh what their what the red team did so i i would say like the best the best defense or what's the what's the phrase uh your best defense is a good offense or something like that so i would essentially just assume that you would use autonomous agents to better do better research for the blue team so that's and break into your own stuff before anyone else does right right so that so that's what i was saying before is that you you instead of like employing a third party company to do pen testing you just you employ an agent to do your pen testing and then you have that company audit the agent or um yeah or or or the or the other way around you have the pen tester uh break into your company and then you have like the ai like maybe work in tandem with the third party so it's just the the defense part is really difficult because at a certain point you're never going to be able to like there's always apparently there's always a way in there is also like the teamwork aspect to defense where like you and i can you know you and i can team up and share what we know about defensive tactics whereas you know so like you know banks institutions infrastructure it's not like they're playing single player as when they're trying to defend themselves the tactics the tactics of the attackers are known right there's a there's a whole set of possible well there's like you have a framework for this you have like the attack uh the mitre attack framework which is just all of the different techniques that are used by the the advanced persistent throughout the the nation state the like like china or russia like their techniques on how to get into um companies or just like you know cyber cyber red teaming um but then like you have like the the common vulnerability exposure I think the CVEs so like as soon as like a CVE comes out I think that I think that's where the AI needs to come in the AI needs to be able to as soon as there's a common vulnerability or like a zero day the AI like you need to have like an autonomous agent dedicated to defending against something like this well I think by the way it's worth I mean look it's it's I think it's both in the cybersecurity thing is both being highly concerned about, you know, science fiction is now, and there's a whole bunch of stuff where it amplifies the offense side.

18:06It's cheaper, multi-threaded, kinds of attacks. For sure. I think one defense is the multiplayer game side. The other one is, you know, the hope, maybe not yet reality, is that we might be able to actually better arm defense with like the best models um you know kind of compute from that you know it's part of like you know what's going on with mythos is not only does it discover vulnerabilities very well but it also suggests you know uh fixes and is kind of at a higher quality model of attack and defense red teaming through hiring agents yeah third parties then you get more resilient to other than the absolute best attackers.

19:00So I'm just like kind of a little bit of like, what are some of the areas that you would kind of say, hey, we have to adjust to an AI universe, but these are the things that we need to be thinking about really making sure we're doing so that we're not just swamped with successful, call it hostile AI cyber agents. i mean honestly that is all that that is like the point of where i i i'm hands off because i honestly like with mythos if mythos is like a purple team philosophy then say what exactly what is what is purple teaming so purple team would be like the mix between red and blue so so you're just it's it's both of them compliant kind of like how gray hat is like white hat black hat that's it's it's in the middle so you're kind of like employing both sides to to understand like what is the the best way to attack what's the best way to defend you know that's that's kind of so if mythos is and i and i i was assuming that mythos was like more of a red team thing but if it's defending and it's doing both right yeah so if it's if it's defending and if it's self-healing and finding vulnerabilities and and patching them up as he as you go then that's that i believe that

20:25Parth Patil:would be the way to go but again like there's also speed right like as the models get faster speed is going to be a huge aspect to this i was reading an article just earlier today on if as long as you can employ you can be like the best at one of those aspects whether it's like speed or you have like the best hardware um it was a schneier on security and they were talking about how ai like the ai assisted evolution is is honestly the way to go that's that's what we've talking about but if you can find a way to have like um implement like so it's i guess it's like continuous integration or continuous deployment if you the next phase of that would be like continuous discovery so as long as you are always looking for the vulnerabilities within the system and you are like proactively yeah proactive ai if you're if you're going that direction then that's honestly the way to go well so let's let's go through you know a couple a couple of the you know key areas that you know very well dozy that the that it's important to do is kind of find okay so you know kind of red blue and purple teaming and kind of having different you know kind of models weaknesses and obviously adversarial review is one of the key things here because um capabilities and blind spots even and just kind of like you know the amazing models fable soul yeah etc so um if we're you know kind of doing a you know kind of like like having to do like this huge surface of different models is the only way to do that to have machines stress test machines does it give some really deep concerns about you know the limits of human review you know what's the what's the way to kind of pull this together both in a way to solve the problem but also in like understanding what's going on yeah i mean i i would say that like you're never you're always going to have to have the human in the loop um ai is good but like ai is still like not better than the best hacker so so again it's it's always going to be a combination of the two um and and and what i was saying before was it it has to so yeah so you in in order to excel in in the space you have to you have to be the best in one of the one of these different disciplines whether it's like the speed whether it's the scale the scope or the sophistication so so that and that's what i was saying before you just as long as like if you can be the first to get to like the vulnerability then sure if you if you have like the technology to scale as far as as far as you need to be like to brute force something we're at the search space yeah right so that so as long as you're like um zeroing in and in one of those disciplines then i that that's honestly where you need to be because you can just be a random script kitty and deploy an autonomous agent and it could it could take down the whole company so that yeah i've noticed that that's the like the script kitty has never been this like it's the same power that gives people vibe vibe coding superpowers is like now it's like there's like casually like you're able to casually like um break things on accident even if you don't even know what's going on so and i guess like when you think about when you think about like the the power of the computer equipped person right like the the parallelization like where do you think like the human like the human like i think my thing is like I like having an agent that just patrols my network yeah while I'm asleep and even if it's just read only it's not not that it actually builds anything but that it's just reading everything and making sure that like no none of the agents on my network are going rogue none of them are like you know putting keys in the wrong places and I like it but I feel that it's not enough for it to just be reading while I'm asleep because I would actually want it to wake me up or I would want it to actually mitigate when something goes wrong like I wanted to mitigate the the um the uh exposure that i that i have there um so i guess like what do you think about like when we give our defensive agents more more agency more proactive like how proactive at what point we're going to start seeing them like you know go find the issue and solve it immediately before you even wake up do you think about that like the human where we as a human in the loop you actually want to not need to be in the loop to play defense yeah i mean i think it's hard to say that like you are you gonna be able to ever just be completely autonomous i don't know i don't know if that's like the best idea because if if the agent starts malfunctioning then it's then it's definitely a problem and that like if you're if you have like a hallucinating like like like adversarial agent that's like that's that that's something that that's something that you don't want right but um it's honestly i i would say that like you have the agent that that patrols you but you just have you just have a second agent that has that specific skill um uh somebody was i was in a room earlier different perspectives different perspectives on playing defense playing at the same time right yeah exactly but i guess the the way to to talk about that would be um skills and not not like human skills i'm talking like the the ai skills if agent skills yeah yeah so one of the one there i was in like a cyber room earlier today and they were talking about like the compounding of skills like um if you have like there are there are several different githubs that are just like autonomously pen testing if if you have like um i think he i think he called it super skills so so you just so you just have like several different skills on top of each other and then you have an agent basically extract from those skills to make like like a super agent i don't know if that that's where he was going with it but that i think that's that's where we're going we're just you're gonna have to compound a bunch of skills together to for an agent to do what you're talking about or you just have multiple agents um with maybe like a few sets of skills and slightly different like theories of how you might want to play the defense right exactly yeah so you don't put everything into one into one bucket right um yeah so i guess that gets into the the white hat space um you know people starting to deploy their own agents to to look for vulnerabilities, look for impersonation.

27:41Parth Patil:It's basically fighting AI fraud with AI defense, which is like a modern version of the old, like of the white hat, the white hat game. Do you think of, do you think that this turns security into an arms race between agents and where the side, where having a better model wins by default? Yeah, so when you say arms race, I'm thinking like who has the most like expensive, who has the most hardware who has like the the like the highest like um capability and like i don't know if that's necessarily the case because you have like instances with like uh deep seek basically blowing it out of the water and spending a fraction of what open ai is spending so i i think it's more of when when whenever there's a paradigm shift that's not in it's not in uh in in arms race it's it's just more of it's it's a new capability or it's it's it's honestly hard to explain i'm having a hard time expressing it but i think it's not necessarily better model but it's like better resourced better right a couple different ways it's an it's like an optimization race if if you will i think that would be the the way to like phrase that well let's come back to one of the things we were talking about a little bit earlier because i do think this is a uh one of the really important thing is to say well offense has has has a attack on you know has an advantage on just finding the one angle that works with the now broader computer surface together with an ability to experiment with it and do things with ai defense has a has the multiplayer you know configuration how do we you know how do we get the incentives to work in defense, you know, kind of coordination?

29:35Because it's kind of like the, you know, there's actually some, I don't want to reveal my weaknesses. I don't want to share my benefits with potential competitors. Or once I share those, maybe those also leak to the offense. How do we get the kind of multiplayer defense incentivized the right way? I mean, the multiplayer defense is, I would say that that's kind of our that's already like what's happening if you have like a company come in and if you like like I said before like if you have company come in to do pen testing they you would have to give them an NDA like they can't like they obviously can't reveal like your vulnerabilities or anything like that to to anyone in the public because that would be a security issue so it's it's more of like cyber security isn't just like the the hard skills of like coding and like cracking passwords or anything like that it's there i mean it's policy is a big deal um a lot the following the law follow having like a very rigid policy on how your company operates would be the way to go but it i feel like the the the law has not caught up to ai yet so there i feel like that's i think that's the advent that we're we're very close to that there's there's just going to be like a like a loophole with ai and and and in some aspect where like the ai the ai that you purchased that you like uh basically put on your own company.

31:20Like you said, like someone you have cited over, they have cited over to you. So now like if it malfunctions or hallucinates or something like that, it's on you, it's not on them. So when things go south, like you publish a report on what happened.

31:36Parth Patil:And then in order to incentivize people to publish anonymized reports, you don't need to give up the customer information, user details, et cetera. But, you know, outlining the style of the attack, because there's going to be novel attack patterns that are emerging, novel or at a higher frequency than they were pre-AI. But when you see, when you can reward people for sharing the, you know, sharing the shape of the attack as it emerges by rewarding them with access to frontier models, for example, like, or rewarding them with access to more defensive measures. And then I think then it's like, it's like, we want a world where you're not playing single player defense, I'm not playing single player defense and then both of us have you know like we have we could have had overlapping defenses where we're recovering you know a wider set of strategies instead of individually trying to to patch everything and i think that creating some kind of reward for sharing sharing information about the new shapes of attacks that are coming online so that the the defenders have a way to like whether that's access to frontier models that can play defense um access to compute to to do that defense fortifying our systems you know okay well you get well let's say one bank notices an attack and then notifying the rest of the you know the financial institutions of that shape of attack before it becomes a bigger bigger problem in the ecosystem right i think it's it's tough to say because i mean there's no there's no way to tell like what someone's true intentions are so I mean not everybody's an altruist not everybody is just thinking that okay well I have compute power I'm gonna I'm gonna with the great power comes great responsibility I'm just gonna do this for good I don't think it really works that way because I mean or I guess you could pay people to make playing defense more lucrative than playing offense I wish that was the case.

33:37Defenders all around the world wish that was the case. Right. Unfortunately, because lots and lots of surfaced, speaking of surfaces to go after, I think we only have time for one more question before we turn to our closing motions. But I know one of the things that you have done some in-depth thought on, because we've talked to you about it before, is one of the most famous deepfake scans that happened in Hong Kong a few years ago, where a finance employee joined a video call with people who looked and sounded like the company's CFO and colleagues, and then transferred$25 million-ish. dollars yeah um tell a little bit about that and then if seeing and hearing someone is no longer verification what replaces it like what what's what where where are the vulnerabilities and the fixes well yeah so the the thing about that case is that um that happened in like 2024 so that was like pretty early yeah within ai so that's so that's like that speaks to like the speed aspect of attacks if you have like new technology like this and you're the first to display it then like like think about like being having a model and or having like sora like in 2023 when you you can't you can't even tell that like no one no one's thinking about that type of thing no one's thinking about deep fakes i mean i mean deep fake deep fakes have been going on for a minute but like the the way to like spin up a deep fake that quickly with like people that you know that you see in everyday life and that you're you're even had you've been snowed so quickly there's no way to like i it's it's insane this uh i yeah so i've been i've been playing with

35:32Parth Patil:a lot of these tools and i wanted to see what the best one of these that you could run on a consumer GPU my own gaming graphics GPU I uh I downloaded I think it was it was it was an open source library and it was the it was called deep cam live it was okay well maybe I shouldn't talk about it but it was the it was it's like the trending open source library of the on github trending and I downloaded it and I realized with one picture of your face I could put your face on in a live video call and and then immediately I just like told my family about it right because I think those are like it's like it's like I am wearing my cousin's face right now and I like put his face on in a video call I was like guys like you should now like this is now the world we live in where real-time video um is you know you don't you can't really trust real-time video and someone can put your face on someone can wear your voice kind of like a mask and all I knew to do was to tell everyone that I want to protect that this is now a capability that it could run on a graphics card that like just anyone has right it's not that it's like an expensive thing it's or that it costs any money at all um but yes so and that was that was like two years ago open source model runs on a at-home gpu i think we get the same thing in so that's the video streaming we get that with voice we get that i mean we have read ai which is hopefully trying to be a positive example of what digital cars can be yeah yeah and i would say like because this is a new form of social engineering right right that that that was that case was like everything that was social engineering that was that was fishing that was it was deep fake it was uh it was a bank heist so that that it that had literally everything you could think of it was whale fishing yeah if if you if you know what whale fishing is um it had so much in it so it was just it's a it's a good lesson for the world to understand like what we're up against but yeah it's just yeah and and then like as far as like like vishing and um like like if someone like steals your like your voice i think you just need to and it's so easy to steal oh that's fishing that's what vishing means yeah voice yeah that's fishing okay wow um but again like yeah it was vishing it was it was a like a deep fake it took like everything but i think you i hate to say it but like you might need safe words for like people in your life right like you need like pineapple or just like something like you need to really interrogate people for the other person you need the other person needs to reveal that they know they are who they say they are without right without uh you know say something that only dozy would know yeah exactly based on your interaction with me based on how you know me it's like yeah the favorite marvel superhero what yeah the new um lantern show there's like this clip going around of like this guy like revealing that he's an alien the guy's talking about voting for obama at the time but then he asked him like who what what school harry potter went to and he couldn't answer.

38:45So that like gave it. So you're obviously not from earth. You haven't seen Harry Potter. Well, Dozie, thank you very much. It's been awesome. I look forward to future conversations. It was an absolute pleasure to meet you. Thank you for having me. Possible is produced by Pallet Media. It's hosted by Ari Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Tanasi Delos, Katie Sanders, Spencer Strasmoor, Imo Zhu, Amon Suri, Danny Garrison, Trent Barboza, and Tafadzwa Niemurundwe.

39:16Parth Patil:Special thanks to Surya Yalamanchili, Sayida Sepieva, Ian Alice, Greg Beato, Parth Patil, and Ben Rallis.