As WormGPT Goes White Hat, Evil-GPT Emerges

0:00Today on the AI Breakdown, we're discussing the darker side of LLM's including WormGPT and EvilGPT. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our YouTube, our newsletter, and our Discord.

0:19Welcome back to the AI Breakdown. Today, we're exploring an issue that sort of lurks right around the corner of all conversations about AI, which is what happens if these systems are used maliciously. We can talk endlessly about AI alignment, as the show on Friday did, and how to resolve issues of LLMs accidentally doing bad things. But what that doesn't solve, as many will point out, is what happens when people purposefully design LLMs to do bad things. This is obviously a growing concern and a growing conversation. For just one example, let's look to the New York Post from earlier this week.

0:58On Wednesday, August 9th, they published a story called Outlaw AI Chatbots Are Making Cyber Crime Easier and More Frequent. The piece reads, ChatGPT might be known to plagiarize an essay or two, but its rogue counterparts are doing far worse. Duplicate chatbots with criminal capabilities are surfacing on the dark web, and much like ChatGPT, can be accessed for a modest monthly subscription or one-time fee. Several dark web chatbots, including Darkbert, FraudGPT, and WormGPT, have recently caught the attention of cybersecurity firm Slashnext. They were flagged for having the potential to create phishing scams and phony texts via remarkably believable images.

1:36One AI strategist told the New York Post that while this type of scams aren't new, the introduction of AI tools for personalization really does mark a huge moment of difference. The expert said, This is about crime that can be personalized at a massive scale. Scammers can create campaigns that are highly personalized for thousands of targeted victims versus having to create one at a time. We have these new criminals that are being emboldened by new language models because they make it easier for people without high-tech skills to enter illegal enterprises. So with that in mind, I was super interested to see this piece on Krebs on Security.

2:10The piece was called Meet the Brains behind the malware-friendly AI chat service WormGPT. Krebs writes,

2:43Krebs goes on. Forums, a sprawling English language community that has long featured a bustling marketplace for cybercrime tools and services. WormGPT licenses are sold for prices ranging from 500 to 5 ,000 euro. Wrote last, the handle chosen by the HackForums user who is selling the service, quote, introducing my newest creation, WormGPT. This project aims to provide an alternative to ChatGPT, one that lets you do all sorts of illegal stuff and easily sell it online in the future. Everything Black Hat related that you can think of can be done with WormGPT, allowing anyone access to malicious activity without ever leaving the comfort of their home.

3:20Now, you'll remember on an earlier episode, we discussed how that security firm Slashnext had analyzed WormGPT and used it to create a business email compromise or BEC phishing attack that was designed to try to trick employees into paying a fake invoice. A representative of Slashnext said, the results were unsettling. WormGPT produced an email that was not only remarkably persuasive, but also strategically cunning, showcasing its potential for sophisticated phishing and BEC attacks. Now from there, Krebs did a little bit of investigating. Quote, A review of Last's posts on hack forums over the years shows this individual has extensive experience creating and using malicious software.

3:57The article points to Arctic Stealer, which was a data-stealing Trogan and Keystroke lodger, another modified version of the information stealer called DC Rat. But in 2021, just after joining the forum, Last told other users that his name was Rafael and that he was from Portugal. Using an account tracing feature, Krebs traced the last user to an initial nickname Ruina Shackers, which when searched on Google brings up a TikTok account of the same name, which is itself associated with an Instagram account for someone named Rafael Moraes from Portugal. Moraes was ultimately reached via Instagram and Telegram and said he was happy to talk about WormGPT.

4:32Moraes said, you can ask me anything, I'm an open book. In that conversation, Moraes said that he recently graduated from a polytechnic institute in Portugal, that around 30-35 % of the work on WormGPT was his, with others contributing, and that so far around 200 customers have paid to use WormGPT. Marais said, I don't do this for money. It was basically a project I thought was interesting at the beginning, and now I'm maintaining it just to help the community. We have updated a lot since the release. Our model is now 5 or 6 times better in terms of learning and answer accuracy. One thing he didn't say is which LLMs had been used to power WormGPT, but intimated that the dataset that it was trained on is significant.

5:08Marais said, Anyone that tests WormGPT can see that it has no difference from any other uncensored AI or even chat GPT with jailbreaks. The game changer is that our data set is big. Marais also gave a brief summary of his own trajectory. He said, My story began in 2013 with some gray hat activities, never anything black hat though, mostly bug bounty. In 2015, my love for coding started, learning C Sharp and more.NET programming languages. In 2017, I've started using many hacking forums because I've had some problems home in terms of money, so I had to help my parents with money. Started selling a few products, not Black Hat yet, and in 2019, I started turning Black Hat.

5:42Until a few months ago, I was still selling Black Hat products, but now with WormGPT, I see a bright future and have decided to start my transition into White Hat again. Interestingly, Marais and the WormGPT project said that media coverage of it has painted it in an unfair light. At the end of July, an announcement on the WormGPT channel on Telegram said, we are uncensored, not Black Hat. From the beginning, the media has portrayed us as a malicious LLM, when all we did was use the name Black Hat GPT for our Telegram channel as a meme. We encourage researchers to test our tool and provide feedback to determine if it is as bad as the media is portraying it to the world.

6:16Krebs, however, writes, It turns out when you advertise an online service for doing bad things, people tend to show up with the intention of doing bad things with it. And indeed, as that has happened, Worm GPT has had to add its own guardrails. For example, they now have a disclaimer that says we are not responsible if you use this tool for doing bad stuff. And Murray said, we have prohibited some subjects on WormGPT itself. Anything related to murders, drug traffic, kidnapping, child porn, ransomwares, financial crime. We are working on blocking BEC2. At the moment, it is still possible, but most of the time it will be incomplete because we already added some limitations.

6:48Our plan is to have WormGPT marked as an uncensored AI, not Black Hat. In the last weeks, we've been blocking some subjects from being discussed on WormGPT. However, despite that, Krebs points out that LAS has still been saying on hack forums and in other cybercrime forums, including Exploit, that quote, WormGPT will quite happily create malware capable of infecting a computer and going fully undetectable by virtually all major antivirus makers. When asked what some of the legitimate or white hat uses for WormGPT would be, Murray said, we use WormGPT to fix some issues on our website related to possible SQL problems and exploits.

7:22You can use WormGPT to create firewalls, manage IP tables, analyze network, code blockers, math, anything. Krebs concludes, Murray says he wants WormGPT to become a positive influence on the security community, not a destructive one, and that he's actively trying to steer the project in that direction. The original Hack Forum's thread pimping WormGPT as a malware writer's best friend has since been deleted, and the service is now advertised as WormGPT, best GPT alternative without limits, privacy focused. Maurice concluded, we have a few researchers using our WormGPT for white hat stuff. That's our main focus now, turning WormGPT into a good thing to the community.

7:56Now, within days of that article coming out, News started circulating that a new explicitly blackhead AI tool had come out as a replacement for WormGPT, which was presumably going soft. The new AI tool was called EvilGPT. From cybersecuritynews.com, a hacker going by the name AMLO has been advertising a harmful generative AI chatbot called EvilGPT in forums. The chatbot is being promoted as a replacement for WormGPT. The post shared on that forum and then copied to Twitter reads, Are you looking for a powerful alternative to WormGPT? do not look any further. I am offering an amazing alternative to WormGPT written entirely in Python for only 10 US dollars.

8:34This is an unbeatable price. The post also reads, welcome to the evil GPT, the enemy of ChatGPT. Now, I unfortunately don't have some really big insight about how to address these threats or these challenges, other than to say that it does seem like the first step is acceptance. Living in the world where we have access to the benefits of LLMs like ChatGPT also means living in the world where that same level of technology can be explicitly deployed for bad purposes. The two things that stand out as really obvious responses to this are one, more emphasis on novel cybersecurity efforts. Notably this week, we got that$20 million DARPA competition around exactly that.

9:11Although that's not nearly enough to actually address this. It feels like it much more needs to be some market incentive. But then secondly, regardless of what one thinks of the AI safety conversation currently, whether it's over dramatized and the risk of human extinction overstated, the clear evidence that whatever the most advanced LLMs that are available for good will also be used for bad should be something we factor in to how we think about releasing or not releasing or controlled releasing more advanced models in the future. Breathtaking insight I know, but listen, I'm just here to keep you informed as we learn about this crazy new world that we're all going into together.

9:48I'm certainly going to keep keeping an eye on this and I will let you know about interesting developments as they happen. For now, I hope you are having a wonderful weekend. I appreciate you listening or watching as always. Until next time, peace.