Deepfakes and the War on Truth with Bogdan Botezatu

0:00Chuck, I didn't know we were gonna do a show on the end of the world. Yeah, it's coming. It's coming. It's coming. With the help of like AI and cybercrime and deep fakes and what hath we wrought upon ourselves? Yeah, well, guess what? Frankenstein's monster. It's looking pretty tame. Looking pretty tame. Bring back Frankenstein. Bring back Frankenstein, baby. Coming up, an exploration of how we're going to go to hell in a handbasket on StarTalk. Welcome to StarTalk. Your place in the universe where science and pop culture collide. StarTalk begins right now. This is StarTalk Special Edition. Today, we're going to talk about scams in the age of AI.

0:52That it's Special Edition means we've got Gary O 'Reilly. Gary, how you doing, man? I'm good. All right. Chuck Nice. I am Chuck Nice and not an AI version of Chuck Nice. AI imitates you better than you faking it. Yeah, exactly. So, Gary. Yes. This topic is way overdue. Oh, for sure. Or maybe it's exactly when we need it. Yeah. Take us in. I suppose scams, if you think of it, have been with us since people started to use money. Oh, even before that. Yeah. You can rest assured that before there was a system of currency, somebody was just like, so that's an interesting bushel of wheat you have there.

1:39A little barter scam. Yeah, a little barter scam. As a matter of fact, one of our most beloved childhood fairy tales is about a guy who trades the family cow for six magic beans. Oh. Which was supposed to be a scam, but it turned out it worked out for him. Lucky Jack. Anyway, if you think about it now, right now, it feels like you can't go an hour without a scam, a text, a call. But how big is this problem? Is it just me getting texts and calls, or is this really sort of a global phenomenon? I'm sorry, I'm going to stop sending you those. Yeah, I wish you would. It's all Chuck. No, but text, phone calls, emails, all of our connectivity into the landscape of humanity.

2:32Right. And now as technology advances, how is that aiding, how is that abetting these bad actors? I mean, tech has been supercharged by AI. There's no doubts about that. And it begs the question, what is real? What actually is real out there? Can we believe what we can't believe? And we're going to break down the deep fakes. We're even going to get into the dead internet theory. And if you don't know what that is, stick around. Oh, I got to stick around there. You really are going to be in for a surprise. Internet theory. And then you'll ask the question, are we all failing daily Turing tests?

3:08What happens if it turns out the internet is all bots? How about that? Some of that will need an expert. So we've got Bogdan Botezatu. This title is cool. I want this on my business card. Director of Threat Research and Reporting. Why I oughta. At Bitdefender. This is a company based in Romania? But they've got offices, they want to protect the world. Yes. And we've got them right here on StarTalk Special Edition. I believe it was Bitdefender that helped Liam Neeson get his daughter back.

3:46How do you work, Liam Neeson? Boda, welcome to StarTalk Special Edition. Hi all, thanks for having me on the show. literally nobody will believe me that i'm shooting a video in such a great company probably people at home will say that this is a deep fake and it's going to be very difficult for me to contradict them now you've told us uh offline that you'd rather go by bob it's easier for everybody yeah oh see that and look at that he says it's easier not not for him but for everybody and by that he means Americans. That's what he means. Because we're some lazy ass name people. Yeah, because people are just like, you know, what is your name?

4:30Bogdan Botezatu. Yeah, I'm gonna call you Bob. You cool with that? You cool with Bob? We are so bad. So Bogdan, give us an idea of what it is globally, and maybe sort of land in the US a little bit more deeply, about the kind of numbers and statistics that we're looking at in the present day as far as scams are concerned. it's very hard to put numbers next to the global landscape of scams because most of these scams go undeported or if they go deported they don't get aggregated globally thing is that there are some estimations gasa places gasa which is the global anti-scam alliance one of the most prominent organizations that deal with anti-scamming, they place scams at inflicting about$1 trillion of losses for 2024.

5:26Did you say T? Yeah. Trillion? Yeah,$1 trillion. Okay. Yeah, trillion. Let me just say, I'm in the wrong business. Okay. It's time to start scams. I have now begun to sit uncomfortably, and it's not the chair. If it's a trillion dollars, are you saying that's the guess or is that that's reported, but most of them go unreported? So that one trillion could be an awful lot more. Probably it's in between. The global cybercrime market is around nine trillion, which means that one trillion dollars for just scamming people would be reasonable. It's very conservative. It is. But the thing is that not all scams get reported.

6:14There's people who are ashamed of admitting they have lost huge amounts of money. Of course. If you look around and talk to these people who got scammed, you'll realize that they haven't lost like$100 or$500. They have lost hundreds of thousands of dollars because these kind of scams run for a very long time. People gain their victims' trust and then they proceed to inflict the maximum amount of damage they can. The other thing is that while probably the FBI has stats for what's going on in the United States, There are so many other countries that are affected by scams which do not report centrally what they have registered in each country or each region.

6:58So it's very difficult to tell how much money people have lost to scams, but$1 trillion seems an awful lot of money lost. You think? Yeah. What are the weapons of choice here for us scammers? I think all of us have experienced some kind of phishing email, but what other weapons are scammers using to get at us? Or tactics as well. In terms of attack avenues, hackers have a huge variety. They prefer instant messaging or direct phone calls because they're very immersive. They can apply pressure and that sense of urgency that makes victims comply faster or fall victim easier. Email is a kind of static means of communication because you're getting the email, you're reading it through, you're pausing a little bit, and then you're like, maybe answering this email or hitting the call is not a good thing.

7:56But when you're woken up at night via instant messaging, hey, this is your bank, your account is being depleted as we're texting, please call us back to find a way to block these transactions. Well, you will be likely to respond to that. So we have instant messaging. We have short messaging. We have phone. We have mass communications because there's a type of scams that goes one to many. We have mass advertising and business social media account compromise. I will detail a little bit later about that.

8:58what's with the phone call where you get it you pick it up and there's silence on the end because that that kind of everybody knows that one yeah yeah so what's the angle there i have two theories uh one is technical and one is a little bit of a scenario that if true we're completely condemned. I started with the first one. One likely chance of these phone calls is technical glitches. You know, scammers use very complex software to spoof their numbers. They use voice over IP gateways to make it look like they're calling from the same country as you and so on. So there's a lot of room for failure when involving this kind of call center grade software.

9:49Sometimes calls hang up, there's glitches that put the speaker, the operator on pause, and so on. So probably there's a technical error that prevents the cyber criminal from getting in touch with you. The other one is, well, I'd say a superstition of mine. um how how do you answer your phone with hello yes maybe no i answer like this who is this sorry go ahead okay fair enough because uh some some people in some in some geographies for instance they will answer with not hello but yes most of europe has yes as an opening line when when you're getting cold. What happens if somebody is building a massive database of words, of yes, of acknowledgments, of confirmations?

10:45Like if I'm answering my phone and somebody records me saying yes to them, where can they play that back to bypass some sort of authentication or confirm a choice of mine? Well, voice is biometrics, right? And sometimes saying yes to something becomes contractual, like it substitutes your signature. What if somebody, a threat actor or a threat actor group might ask for confirmations from people? No, that makes sense. So what they're doing is they're capturing your voice. Yeah. If I told people 10 years ago that based on a two-minute conversation that we had on phone, somebody will be able to spoof my voice and impersonate me everywhere for tens of minutes or hours.

11:33Would they have believed me back then? So Bogdan, looking at that aspect of it, if you've got new technologies, which we know are evolving rapidly, if not quicker, how do we get into deep fakes? How prevalent is the deep fake scam now on the landscape? They're very prevalent and they're making most of the victims. I was telling you that I'm clustering scams on a one-on-one type and on a one-on-many type. One-on-one scams are those that happen in instant messaging where you're getting approached by a stranger and they try to earn your trust and then they will guide you towards some type of scam.

12:20There's a one-to-many type of scam communication that is massively aided by deepfakes. cybercriminals are building deepfakes with people that the world recognizes and trusts like you folks right you are online influencers cybercriminals have a lot of footage with you that they can use to train algorithms and people tend to listen to you because that's what they do with key influencers there's also politicians doctors who are very famous and they become the base of deepfakes. With these deepfakes, cybercriminals start promoting all types of scams from medical supplements to huge crypto investments, you know, you name it, they have an opportunity for everything.

13:09And these deepfakes get broadcast either on stolen YouTube accounts or on social media posts that are boosted by paying for advertisements. They use the trust given by the figure that has been impersonated, and they are using large channels to reach huge audiences. And from there on, of course, some people will fall victim to the scam. They will heed the call to action, which is normally visit this page or call this number and sign up for this opportunity. So this is how deepfakes work. We took a look at what's going on now and we see that there are tens of thousands of such ads running on social networks.

14:01There are large YouTube accounts that have been compromised and used as a billboard for crypto scams. One of the largest accounts that has been compromised had 28 million subscribers. So when hackers got the hold of that account, they were able to broadcast that deepfake to 28 million potential victims. That's more than Romania has population. Wow. So can I, if I can, if you can indulge me, I'm going to, I want to tell you that when I fell for a deepfake, I actually fell for one. And I'm embarrassed as hell. But it was very sophisticated. So. Which you have to say because you fell for it. Why you gotta hurt a brother?

14:45Why you gotta hurt a brother? Did I set your lawn on fire? I'm just saying. Damn. You can't say this was a simple deep fake. I'll let you be the judge of whether or not it was sophisticated. All right. Okay. So here's the deal. There was a deep fake of Sam Harris, who happens to be somebody I respect. All right. Okay. How they knew that, I don't know, but it came into my feed. And he was touting a very specific kind of product, not a brand, nothing, just a kind of product. Okay. Yeah. So I looked it up and of course they have, you know, they have your search history and all that kind of stuff.

15:28Right. Yeah. Yeah. So then I received a very specific ad for the product. Okay. So deep fake. Right. I respond with just a search, and then the search responds back to me with more information. And then over the course of this back and forth amplification, I bought the product. You dumbass. Okay. So, Bob, what you've just described there. All right. I knew he was going to do it. You're just lacking some empathy, Neil. I'm sorry. Enough with this man here. Go. Is that a common template that you're seeing with a deepfake? This is AI going full circle. So the AI is building the billboard that will sell to you.

16:15The AI algorithms on social networks will know how to profile you and what ad to serve you for maximum efficiency. And then from there on, you will be chased by ads, all pointing to the same product until you're ready to shop. So cybercriminals work most of the time like corporations. So they have their own product division that builds the deepfake. They have the translation division that builds the multi-language content. They have the web dev team that keeps the servers running for the scan pages to reach you. And they will have quality assurance and sales support. You mentioned before that we partner with law enforcement.

16:58Yes, that's something that we normally do on high profile cases and scams are some part of these law enforcement corporations. What we learned about is that these cybercrime businesses have call centers that take people's calls and sign them up for various stuff. people in uh employed in these call centers are screened uh before employment with light detectors to make sure that they're not undercover cops to make sure that they will not betray the call center's cause and so on so this is cyber crime incorporated it's not a scam business ran out of the somebody's basement it's business that cyber criminals have invested money in order to make more money.

17:51Wow. Wow. I mean, that's infrastructure. Is there a specific demographic that these organized scammers are looking at? Is it a gender based? Is it an age group? Is it geographical? What is it? Or is it just, you know, what will take anyone's money? We don't care. Is it a black comedian who co-hosts a podcast? I hear that's a very popular demo with the scammers. No, in the end, everybody's welcome to put their money on the table and leave it there. That's perfectly fine with cybercriminals, but they have various approaches because they don't have a scamming syndicate yet, right? They're not unionized in a way that would allow them to organize in order to target demographics, right?

18:39So what they will do is find out a local scam that converts well, that depends where you are, right? In some places of the world, for instance, leaking out your social security number is huge, right? And will bring you a lot of hurt in the foreseeable future. In Europe, for instance, some parts of Europe, leaking out your social security number doesn't have any value. It's pseudo-publical, actually. So cybercriminals are looking for information or types of scams that convert well in the region. They don't target demographics, but they are focusing on specific aspects. Some of them are focused on romance scams, for instance.

19:24They will target men more than women because it looks like men are much more... Horny and desperate and lonely. No, careless when it comes to sharing information with partners, right? Women are a little bit more reserved. They don't go as fast and as far as the male population, but they still, when they fall for the scam, they fall the hardest. To answer your question, I wouldn't say that cyber criminals are targeting demographics, but rather that there are specialized cyber crime groups that prefer one type of scam over another. And us getting targeted by so many scam groups on a daily basis would look like there's something very structured as the same organization targeting different demographics with different tactics, while it was about us getting targeted by multiple cybercrime rings at the same time.

20:28So business is good. That's basically what you're saying. But we're in a different space there because people would not necessarily do a deep fake to extort who and what we are, but they can deep fake our integrity, our name, our authenticity. And there's one case where someone just scripted this narration about the Big Bang. It was like 85 % correct. And I got fully deep faked into being the narrator, the person speaking those words, showing me in a podcast setting. And it went online and it got boatloads of views. Well, there's the money. Well, I guess, okay. So there's the incentive. Okay. And even a good friend of mine, Terry Crews, who is himself an actor and a public figure, he texted me and said, Neil, this is great.

21:27This is great what you did here. And I said, what? And I looked at it and said, that's not me. That's not me. And a funny thing, I don't want to say this publicly, but maybe I could or should or would it matter? When I speak, my words have way more rhythm to them than that deep fake did. So I'm just saying I know me when I'm speaking. Right. And when somebody is not me speaking, even if they're using my word, it ain't me. Anyhow, it fooled him. And so this. Also, the deep fake sounded like it was on helium. You know, when you look out into the universe. It was missing some of the timbre of the lower registers of my voice.

22:09So my, my, my only point there is, so yeah, 15 % of it was either misleading or wrong. And there've been others where just, just as Chuck was duped by a deep fake of Sam Harris, there was a deep fake of me commenting on a video game release and that people thought it was real. and it was almost comical, but it was, so. This is Neil deGrasse Tyson. I too like sitting in my mother's basement.

22:42As I'm playing this video game right now. So what do we do? Do we call you, the companies like you? What happens? And are we a lower priority? Because no, they're not draining our bank account yet. Where do we fit in that spectrum? My guess is that what you described is a crime that has two distinct victims. The first one is you because you have just become an unwitting accessory to a bigger scheme that was shown to a potential public. Your reputation is damaged here and that somehow can be controlled because you have the leverage to report that video to the hosting platform and probably take it off.

23:27but you have still presented some information you that version of you has presented some maybe misaligned information to your potential audience and that's that's how deepfakes normally run cybercriminals pick up a very prominent figure like a president a bank governor a medic And then they place a discourse on top of the video. They will attempt to convince people that what that person is saying is true. The people will flock to heed the call to action and probably will lose money. So for some people, there is the reputation of damage. That's you, the persons that get impersonated. for some other people is the financial loss that they have caused themselves when they heeded your call no you don't call uh bid defender for that you call the platform and have the video removed you use your outreach to tell the people that you're being impersonated and they should do their due diligence.

24:44And you also might want to educate the users, which we're actually doing right now. We're talking about deepfakes. We're talking about the possibility that everybody can create an online version of us with different hidden agenda. And I think that this educational part is the most important. Speaking of education, Are there telltale signs that you're looking at a deep fake or hearing a deep fake? Now, Neil said that the cadence of his speech was kind of a giveaway to him when he saw it. But are there things that we can, as laypeople, look at in a deep fake and say, oh, if I see this, this and this, most likely or definitely this is a deep fake?

25:30And you tell that to the deep fake and next time it doesn't do that. we're losing game yeah we are but are there right now that that we know of i would say yes yes and no uh for starters there are a couple of telltale signs like maybe poor lip synchronization or some sort of artifacts introduced by the ai some if you remember a while ago the ai used to have a very difficult time aligning teeth or representing the amount of fingers but that changes in time that as technology evolves these things get perfected and what i'm trying to say is that we should rely less on technical artifacts or tell the other science to tell a scam and focus more on the likelihood that what we're hearing and seeing is real i saw the impersonating videos that neil sent over and when we analyzed them we focused on let's say a couple of key elements that will demolish the story like neil neil is a very knowledgeable person in the science field he wouldn't spend much time commenting games right he wouldn't use that language he wouldn't be recommending products he would not do that probably we are going to need that then upcoming versions of our technologies uh for fighting scams to include deep knowledge about public people or the most prominent people in the world that are likely to fall victims of impersonation and create some sort of a what would that person do recommend speak about discuss publicly and so on very very important fact because that video game review it had a lot of vulgarity in it right and i'm not a vulgar guy yeah i'm not that guy plus I don't sell anything either.

27:28I don't sell anything. Right. So if you ever see like, hi, this is Neil deGrasse Tyson for delicious Buffy Bison beef jerky. You know, it's not real. It's just not real. Yeah, there's even pressure for me to sell things for the ad spots of this podcast. I don't do that. Gary and I are the whores that do that. You're welcome, Neil. you're protecting my thank you this is our camp we are here on these streets that opens up a lot of opportunities because whenever you're misbehaving or do reckless things you can say just you know it's an impersonation i'm not doing that normally right okay bogdan we've seen and you've explained it brilliantly thank you about the development and use of technologies to bring forward different levels, different types of scams and deep fakes.

28:27How much of this is pre-planned psychological attack on victims? And how are scammers building a psychological aspect to this? My theory is that 90 % of the scams are psychology and probably 10 % technology and science. That's because scamming people is actually hacking into their brain, pushing some buttons that generate emotions. Every type of scam that we have analyzed has some sort of psychological cues that cyber criminals want to pick up. Let's take failed package deliveries. You become curious about where did the package come from and what might be in it. That's enough for your brain to switch off the rattling sound that says, hey, probably that link you're going to follow will lead you to a phishing page.

29:26The brain no longer listens to these warning signs. You have romance camps where cybercriminals are exploiting and preying on the lonely. They don't target people who are using technology. They are targeting people who are feeling lonely and they are feeling so lonely that they will be willing to spend all the day talking to a stranger who inadvertently sent a message because they misspelled a phone number. We have cybercriminals that prey on the natural greed that people feel, like get rich quick. Now, would you like to multiply your money 10 times? they're again they're they're not trying to demonstrate an economic impossibility they will try to push that button that says hey i need more money because that's the human nature so probably most of the scams that we face on a daily basis are psychology and technology just widens the net makes cyber criminals more effective makes them capable of targeting people what in a different geographic regions speaking a different language.

30:39A couple of days ago, I got texted by a scammer on an instant messaging platform, and they wrote the message in Romania. And I do what I usually do, answering Finnish. Finnish is a very niche language. There's like, what, four million people speaking an almost impossible language that it's very difficult to understand. And that's, you know, that's my tuning test if you're able to reply me in finnish you'll abort right guess what they did they removed the first message and replied in finnish and they carried a conversation for a couple of messages sometimes they would divert back to romanian uh delete the message and then replace it with the finnish translation and they would do that in almost real time so what i'm What I'm trying to say here is that technology is an enabler for them.

31:37They are using the same psychological patterns that I talked about, but now they're able to cast a wider net because they have APIs to mass mail, mass communicate with people. They have real-time translation to help them address markets that were impossible for them. And they have huge opportunities on the payment scale because credit cards are universal. And if credit cards don't do the trick, then probably you're going to have to exchange real money into Bitcoins or cryptocurrencies. And API stands for what? Remind me. It's Advanced Programming Interface. It's a way that you can hook up, let's say, an instant messaging application to a computer to mass communicate with thousands of people at the same time.

32:26Wow. So I think it's, you know, I may sound cliche when I say this, but this is what I was taught my entire life. That one, you don't get something for nothing. Two, if it's too good to be true, then it's not. And three, and this is the part that's very hard. Don't ever want to believe something more than you want the truth. because if you want to believe something, you will discard everything to get to your, to see your belief confirmed. You left out a fourth one. What's that? Don't be a dumbass. That's my rule. Those were my parents' rules. None so blind as those who refuse to see is kind of like another way of rephrasing that.

33:17And talking of phraseology, I think we, just us three here, I've learned recently some scamming language. Firstly, I'll ask you to break them down. One is honeypot. The other is pig butchering. So which one? Well, one sounds pretty good. Yeah. And it ain't the honeypot. So Bogdan, could you break down each phrase for us so we understand and gives our audience an understanding as well? I like the way you have split them into adversary language and good guy's language. I'll start with pig butchering. This is a type of scam that has been going on for quite a while. It's very popular in Southeast Asia.

33:59That's where it got its name from because it technically means fattening up the pig before you sacrifice it. And that's what cyber criminals are doing to the victims. They gain their trust. They keep conversations going on for weeks, maybe months trying to gain their trust and get as close to them as possible and when they earn their trust they're going to create massive financial losses because they already have that person's trust a very common uh big butchering uh type of attack is somebody texting you normally with an uh with an opposite sex handle like if you're a man they will impersonate a woman And they will be asking you, hey, this is Jennifer.