#319 Subho Halder: Why Traditional App Security Fails in the Age of AI

0:00If an app worked, it looks good. that was considered a success. But what stood out is how much trust we placed in this mobile application. I mean, today, if you look at it, all your credit card information, all your personal identifiable information are there in your phone. And you are carrying these credentials, payment data, personal identity, and behavioral information everywhere. The offer started developing much more faster, and it didn't stay still anymore. Release cycles started compressing. It's no more months. It's gone down to weeks and now even days. APIs have multiplied. Too many APIs are in the world.

0:40This episode is brought to you by Tasty Trade. On Eye on AI, we talk a lot about how artificial intelligence is changing how people analyze information, spot patterns, and make more informed decisions. Markets are no different. The edge increasingly comes from having the right tools, the right data, and the ability to understand risk clearly. That's one of the reasons I like what Tasty Trade is building. With Tasty Trade, you can trade stocks, options, futures, and crypto all in one platform with low commissions, including zero commissions on stocks and crypto, so you keep more of what you earn.

1:27The platform is packed with advanced charting tools, backtesting, strategy selection, and risk analysis tools that help you think in probabilities rather than guesses. They've also introduced an AI-powered search feature that can help you discover symbols aligned with your interests, which is a smart way to explore markets more intentionally. For active traders, there are tools like Active Trader Mode, One-Click Trading, and Smart Order Tracking. And if you're still learning, Tasty Trade offers dozens of free educational courses, plus live support from their trade desk reps during trading hours.

2:14If you're serious about trading in a world increasingly shaped by technology, check out Tasty Trade. Visit tastytrade.com to start your trading journey today. I'm going to myself. Tasty Trade Inc. is a registered broker-dealer and member of FINLA, NFA, and SIPC. So Subho, can you introduce yourself to listeners? Thanks, Craig. It's great to be here. My name is Shubo Halder and I'm the co-founder and CEO at Appnox. So I started my career as a security researcher long before AI was a part of everyday conversations. But back then, I spent most of my time reverse engineering mobile applications for banks, telecoms, and other big consumer companies out there.

3:11And mobile security at that point was largely ignored. If an app worked, looks good, that was considered a success. But what stood out is how much trust we placed in this mobile application. I mean, today if you look at it, all your credit card information, all your personal identifiable information are there in your phone. And you are carrying these credentials, payment data, personal identity, and behavioral information everywhere. But still, security team kind of treats these applications like a thin client, assuming the real risk actually stays in the server right and that disconnect is what led me to start app nox and we started app nox almost 12 years ago and the idea was very simple security has to move fast and at the time most security testing was manual slow compliance driven and you would test an application once or twice a year.

4:09Generator report hope nothing serious has changed between releases. But unfortunately software started developing much more faster and it didn't stay still anymore. Release cycles started compressing. It's no more months. It's gone down to weeks and now even days. APIs have multiplied. Too many APIs are in the world. A lot of third-party SDKs exploded and mobile app started becoming more and more like a living system. It's no more like a static product which is there. And over the last decade, I watched how the security landscape evolved as well. Applications are no longer passive. They learn from user.

4:52They keep adapting over time. It is now increasingly behaving as if how things are changing. And from the security perspective, that's a completely new challenge. And historically, if you look at it, security was about protecting code, infrastructure, building perimeter security. But today it is more understanding about the behavior, the intent, the outcome of the system. What we are no longer asking, is this code vulnerable? What we are asking, is this system doing something it should not be doing, even if the code looks really fine. And that's where many organizations are struggling. Their security models are built for a slower, more predictable world, a world of all static binaries, fixed logic, clearly defined boundaries.

5:41But with the advent of AI where anybody can now build a mobile application, I mean, you can go to any website right now today and build a mobile application free of cost. And it kind of breaks those assumptions once that becomes a reality, which is a reality as of now. When we talk about AI and security today, I don't see it as an incremental improvement. I see it as a category shift and we are moving from securing software as an object to securing software as a living system. And that's the lens which I want to wear today and have more discussion today. So, yeah. Yeah. And with this, actually, I have a son who's working for an AI app.

6:23The problem is malicious actors that are creating apps that do bad things and putting them out there and people downloading them unsuspecting. Or is the problem apps, AI-based apps that have security flaws that bad actors can exploit? Which are you focused on? We are focused on security of the AI applications which Bad Actor can focus on and I'll give you some really basic insight. So if you look at any applications in your desktop today, your Chrome, like if you open ChatGPT in your Chrome or Perplexity in your Chrome, I mean you can use their desktop application as well. But let's say this application is accessing your location information.

7:14Now, Chrome has its own permission system, which kind of pops up saying that, hey, do you really want to share your location? But if you look at applications in mobile, those are a little bit more restrictive. Like while you're installing the application, it kind of tells you these are different permissions which this application requires. And it's just a one-time grant. And you don't care about it once you go out and install applications, right? Now, GPT in mobile application does have access to your location. but you had no idea about it unless you actually go inside settings, go inside the application and figure things out.

7:51The reason I'm saying this is not because ChargerPity is a problematic app, but the reason I'm saying this is because any actor, any attacker can actually create a malicious ChargerPity wrapper, keep it for free and say, hey, you don't have to pay a monthly cost, like a monthly subscription for ChargerPity. Why don't you use this for free? and you go ahead and download that application from these stores and then you realize that oh yeah i'm able to you know use the chat repeat models but behind the scenes these applications are actually sending so many sensitive information and if you compare your desktop with your phone holds much more very sensitive information to you it has your financial records it has your credit card information you pay your grocery stores using your phone you know using your devices nowadays with you know apple wallet or google wallet it has all your ssn certificate ssn information it has your healthcare information and so many more right so now these applications certainly have an access to such incredible data pertaining to you right and just imagine what happens if such application has suddenly access to those data and a malicious actor can actually steal those data in it.

9:13In 2025 starting, we started this thing called finding out, figuring out fake applications in the stores, in the app stores and play stores I'm talking about, not third-party stores. And what we found was we found a couple of applications which are there in the app store and play store, which we helped them to take down those applications, which was not a legitimate application. It had the logo of, you know, Chagipity. It had a logo of WhatsApp. It had a logo of valid brands. And it talked about the same thing, but internally it was leaking so many things internally. Yeah, and we were able to find or figure those applications out.

9:56And thankfully we were able to figure those applications out with the help of AI itself internally, right? Because again, at scale, security at scale is where AI kind of wins the race. Unfortunately for human at scale, you know, you still need time. Time is the most complex complexity in terms of doing a security testing. So that kind of happened. That kind of helped us to start validating this idea about figuring out fake applications in the app store, play store, or even in third party stores and figure things out. So that's where we kind of focus on. Now, the other part of the focus is where we cannot focus on it is, but I'm hoping OEM providers like Google and Apple does, is to educate the users themselves to make sure that they don't go ahead and fall a victim or a prey to actually downloading those applications.

10:47So that responsibility lies with the OEM of the vendor, unfortunately, but all we can do is making sure the OEM platforms, the marketplaces are secured enough to make sure those fake application doesn't go inside those application stores. Yeah. And how do they get in the stores? Because presumably, I mean, I've been through the app submission process with Apple's app store, and it's fairly rigorous. Is it because the app reviews are now being done by AI and they just miss this or or is somebody that Apple not paying attention yeah I think that's a very good question so just to understand what is the motivation for these marketplaces like the play store and the app store to publish applications their motivation is to make sure that it does not actually go ahead and affect the users which means they look for if the application is not acting or in the classic security world we call it a virus or a trojan attack it basically checks for that it checks if the application is not doing things which it shouldn't be doing but then these fake application which goes into the store it is a very benign kind of an app it does not actually affect, it does not actually hack into your account and do a payment for you.

12:18But it is more of a data farming kind of an application. So there are a couple of different kinds of fake applications which are there. So which we had defined. One is a simple wrapper kind of an application which are more towards focused towards earning ad revenues. So I'm going to, let's say I'm going to say, hey, charge a little light and I'm going to try to you know put it in a play store and an app store or in an alternative store and say that hey this is a valid application it's a benign application it does gives you it does work it's just that it's an ad where a lot of ads are there and just so that if you use it i'm going to give it for free just you need to click on the ads and i'm going to earn some money out of that so that's one one category of fake application the other category of fake applications are application which actually do data farming.

13:09It tries to gather as much data as it is from a device. It's like it'll download all the contacts, it'll try to figure out what are the wi-fi it is connecting to and it still seems benign. So even for a Play Store and App Store review process, it feels like yeah this app actually requires a camera access to actually click pictures so that you can send it to the AI to analyze image but behind the scene it is like hey I'm not only going to send this image to the AI to analyze it, but I'm also going to send this image in my own server and I'm going to just create a profile of whoever has installed my application.

13:47And the third one is the most dangerous one and that is the one which most of the Play Store and App Store kind of tries to look for, where this application actually is attacking the users who are using it. They're trying to actually succumbent the protection the OEM has placed into the devices And those are the applications which gets flagged more often than not. If you look at some of the data, last year to 824, and it has nothing to do with AI, the most fake application from a category is a banking application, right? These are all applications which gives you loans, which kind of, you know, you download this application, you can get a quick$10 ,000 loan just before you get your salary paid.

14:30And these applications have access to your SMSs, has access to your phone book and address book and everything. The reason they state the reason is because I want to build a credit score, how much of the loan can I give to these people? But just imagine if I come up with another application mimicking a similar behavior, but I'm not going to use those data to actually give you loan, but I'm going to do it for something completely different. And that's where Play Store and App Store review process kind of fails in there. And I mean, if you look at the data every year, almost 200 ,000 to 250 ,000 applications are taken down from both App Stores and Play Store for a reason.

15:17Almost 50 to 60 % of the reason is it is violating the safety norms for those devices. And that's why they have been taken down. Yeah. So the people putting those apps up, they're harvesting data. What's their business model? Then they just sell that data to data brokers or is there some other use? Absolutely. So these data is having sold in data brokers, these data is having sold in dark web as well. So again, as I mentioned, there are three kinds of fake applications, right? You have the Adware application that is completely revenue model where I want to ride the hype of whatever the new launch which has happened to make sure as many installs can happen and I can earn some ad revenue out of that.

16:06So that's the number one kind of application. Second is the data farming application. Now these applications, they go ahead and sell the data to data brokers and they kind of try to sell it to competitors. Like I'm sure if OpenAI has launched something today, then Anthropic is also trying to look for getting the user base of OpenAI to come and install their application, right? So there will be data brokers who will be like, hey, you know, I know how many people have used this, Why don't you use, you know, use kind of this or things like even these data are also sold to banks and financial institution to target people who has better, who has a lot more spending capacity and power so that you can target more ads and you can reach out to them and have a lot of cold calling in progress to do it.

16:51So that's the second type. And the third type, as I talked about, those are very strategic kind of attacking the user itself. It's far more like a malware or a virus in the cybersecurity domain. That's what we call it. And that, that is a very malicious intent that has nothing to do with earning money or anything, but to do damage and harm to the user. Yeah. Yeah. And these are apps that are engineered for those purposes. But what about, I mean, AI is being embedded into everything, all apps now. And the apps are starting to include agents, intelligent agents that adapt and evolve as you use them.

17:33So how much of the problem is, for example, I build an app, I have an API call, it's a quote unquote AI app. how much of the problem is just I'm not careful in my security and that becomes a vector for bad actors to reach the app users that's a great question actually so the how we should look at it is two perspective to this it is AI for security and security for AI that's what I keep talking to people as well. So AI for security is how do we leverage AI to make sure the security of the applications are okay. And the other way around is how secure is the AI itself in terms of acting on those devices.

18:29What we have seen is in terms of security for AI, whenever we are embedding this app embedding this ai models into these applications what we have seen is it requires a lot of testing in these models to make sure that it these agents do not go haywire and do not start collecting data which it should not be collecting data on and things like that and we have seen a lot of instances where i mean recently this came out and it has nothing to with mobile, but this has come out that cloud code, like I love cloud code, right? So cloud code has started, I mean, there was an incident where cloud code went ahead and started deleting sensitive directories, right?

19:12How do you prevent agent to actually stop doing dangerous actions on behalf of the users, right? And that is a very important thing. In the mobile realm, this is all the more important because now we have this set of very sensitive data and then you have AI which has access to these very sensitive data. How do you put in guardrails in the model and the AI to make sure that you know even if they have access to the data they know exactly how to access it or you should not access those data and then whatever the guardrails are. So are those guardrails good enough to make sure that doesn't happen.

19:51The other way around is actually using AI for security. A decade ago in cybersecurity world, there was a term called script kiddies. Script kiddies and nothing but who are entering into the cybersecurity world. And they see some script by which you can hack into a couple of things. They just run the script. They don't understand anything. And it kind of tries to get into the system and they are like in hacking or in a vulgar word we say noobs in security people who kind of does that and now that has changed completely right you can use this ai model you can use cloud code to actually go ahead and try to hack into system you can instruct a cloud code agent to go ahead and say hey cloud can you just observe this android application to come by this application and figure out if there are issues and or things like that the power of ai is the reasoning cycle the how it reasons out the situations, right?

20:50And that changes everything. So now these script kiddies who are also called noobs during our time are no more script kiddies now because now they are more like a prompt engineer or a security prompt engineer. Now all they have to do is make sure they instruct the EM or to do something which is very offensive and which goes ahead and does it on behalf of them. And that kind of changes because the attacking part now has become so easy. The barrier to entry is so less that the defending part has become more and more problematic. That's why we come in as well. Like at Apnox, we are the defending side of it, right?

21:29We have to cover all the bases, right? Even one single point of failure is problematic. For attackers, they just need to find that one single point of failure. They don't need to do everything. So for a defending kind of a situation, it becomes very difficult to use the old methodology which we used to do it in the cybersecurity world. It's no longer those algorithmic way of figuring vulnerabilities out, but it's more about using the AI, the reasoning cycles to actually block these attacks which are happening by attack. I'm sure that this world is not going to be too far where you're going to see two AI agents trying to you know fight against each other one trying to defend the other trying to attack and that's going to be a reality pretty soon so yeah that's what my view is with this yeah yeah actually that's fascinating the idea of of ai agents sort of fighting a defender and an offender the how is trust becoming a measurable performance indicator for ai systems i mean what will companies be expected to prove to users to build trust?

22:39I think one of the biggest problems with AI is it's like a black box. You feed in some information, it gives you some information out. And what people are really worried is, I really don't want to give my personal data to AI because I'm not sure how this data is being processed internally. Is it been saved somewhere? And how is it being, you know, processed and giving me an output? And that's where the word trust comes into play. Trust is very important in an AI world, right? So there's this very thin line where, you know, we are a trust implicit society, which means by default, we try to trust something, but if the trust is broken, that's it.

23:27Then we become a trust explicit society. then we are like, we're not going to trust you. First show me some proof and then only I'm going to go ahead and do it. And that's the kind of a society which we live in in today's world. Unfortunately, there has been certain instances in the past with AI models leaking information or getting into datas, which they should not be allowed to have access to in that kind of built that kind of built this behavior that hey I do not trust AI models so it becomes a very big problem for companies like us who are utilizing AI to power or to harness defending part of it to actually build trust for the users and how we build it is actually by providing transparency And I had talked about it previously as well.

24:21Transparency is the key to building trust. If I transparently tell the users, this is how I process your data. This is how the AI processes your data. And this is what the outcome of the data processes is where you start building trust. That's the initial first step towards building trust, right? Right. Incidents like, if you remember in the start of 2025 incidents, like DeepSeek, you know, getting all the data into some servers who people were not aware of. A lot of news came up because of that. A lot of places people stopped using DeepSeek altogether. That kind of showcases that how important trust is.

25:04Right? Somehow we do trust Anthropic and OpenAI to make sure that our data is there because of data residency, data privacy, and things like that. Okay. My data is not going out of the country. It's still there, which is great. But whenever the user gets spooked, if the data goes to some other places or if the data is being mishandled or misused. So trust is the very important thing in terms, in the AI realm, and it is very important for companies, not only like us, but companies who are in the AI space to actually build that transparency in terms of how the data is being handled and how AI models are using these data internally.

25:42Yeah. Yeah. Actually, that's interesting how not only because of AI, but certainly AI plays a big part of it. That trust is eroding in certainly in US society. but globally, trust in technology, trust in media, trust in governments, trust, you know, and this is a big part of it. So how do companies start evaluating not just who's accessing their app, but what their intent is? So it all depends on how the data is being processed and what is the intent of the data which is getting processed it starts with that if a company is actually utilizing your personal data to to predict or to do something for example let's say I'm a healthcare company and I really need access to your healthcare accounts to actually you know predict something or to help you something that is something which is a noble cause right and I should be given a control about what are the data which I should be sharing and what are the data which I should not be sharing right so companies should be giving those kind of controls to the end user and should also convey what are the data needed to actually make a proper work the use case of it right until unless I know the use case of it I may not be trusting the system to give those data out.

27:22I understand like for a health, it is a big problem in a healthcare kind of an industry because they need access to all your healthcare to actually give you a proper treatment plan. And the only way that can happen is if you can share this data with the company. Now I'm not talking about AI, I'm just talking in a general mindset. And then there are certain controls as well. So there are things like data scrubbing. Sometimes you need to share data, but that does not need to be involved in terms of linkages. So there are things like data lineages and data linkages. So for example, you can share your health records, but that does not need to be identified that Craig's health record is this.

28:06It can be like person X and this is the health record of person X. right and that is enough for a model or for the company to actually predict or give you a proper treatment plan depending on what of the data it is it is easier to explain it in a healthcare realm but you can extrapolate from the healthcare realm to other realm as well like your payment processing your consumer brands your e-commerce everything you can just just extrapolate it to other such industries as well. So it is very important for users to understand why they need the data, how much data needs to be shared, and what is the minimum data which needs to have to be shared so that you know they get the benefit out of it at the end of the day.

28:53Yeah, you know that the problem is you mentioned is educating the public because GDPR is as I see it has been largely a failure because in order to comply, you know, companies put up this little, you know, accept button and nobody has time to read through. I mean, the fine print, you need a lawyer to go through it. And so they just accept. And with this, as this trust issue grows, how do you educate people or companies as to how they should handle this stuff. And also, you were saying how, you know, we all, not all, but we trust OpenAI and Anthropik because the sort of core researchers are at those companies.

29:53They've raised a lot of money from reputable VCs and investors. but I don't know what open AI is doing and I use it every day you know for a while I know they're you know and we've all had that experience of having a conversation with our wives about or spouses I should say with about you know going on a vacation to the Bahamas and next thing you know we're getting fed ads about vacations to the bahamas and it's like the go-to is that my phone is listening to me i don't know maybe it is i always tell my wife no it's these systems gather so much data they can triangulate you know based on something search that somebody in the family did but but it does make you uneasy how how do companies build that trust and how do how should consumers react to these apps yeah i think this is a great question and the thing is trust is a kind of a mix between the law the government and the company right it has always been a mix of all the three because just as a like I run app knocks and I can just tell you in a blanket statement I don't store anything in my database right why will you trust me if i say that right it will only trust me if there are certain processes in place so now if i back it up by saying that hey i have sock 2 type 2 as a certification which kind of puts in a process or a control which kind of tells me that even if i hold some sensitive data in my databases i still have some access control to it but still you will be like that's okay but But I still, I mean, there's a little bit of build of trust which happens there, but then it's still, it's not fully trustworthy.

32:12And then the third part is the law of the land, right? How stringent the law of the land is. And you talked about GDPR being not, not a very strong privacy following. And I agree with you, like those accept cookies, which comes in. Now our brain always is programmed to click on accept, but have you tried clicking on decline the website still works normally right it is just a consent it doesn't matter whether it's an accept or a decline unfortunately that's the truth in today's world right if a company is actually building on trust if you actually click on decline it may be some of the videos let's say a youtube video is there to showcase how how the product works but if you click on decline that YouTube video should not be loading because otherwise you are consenting that this, your data will be shared to YouTube and you do not consent to that.

33:08It's a very simple way of figuring out whether they have actually implemented it properly or not. Unfortunately, in today's world, all we need to see is just a disclaimer message and that's it. But still it builds a little bit of trust. And the third thing is the government. I'm happy about the US government. they do have a con congress where they if there are issues which happens they do call the companies out they do hold them accountable i think that is a really good thing that kind of builds public trust right so the question over here is why did deep seek which is a chinese-based company the servers are in china versus open ai why do i trust open ai more than deep seek the reason is i know that today if something terrible happens with my data unfortunately if something terrible happens only with my data maybe there's nothing i can do about it but let's say a terrible thing happens to the whole country with a lot of data there's a mass problem which happened the government is there to call them up and you know they can ask them questions but what happens if deep seek has the same kind of a problem can the u.s government go and ask them hey can you come here and answer this question that's not going to happen and will the chinese government do the same thing over there in china saying that hey why is this happening people do not trust that so it has nothing to do with whether u.s government is better or china government is better but it is about what is the process how do people perceive these processes are in in a broad daylight right and these are all psychological matter unfortunately this has nothing to do with tech this is everything to do with psychology.

34:49That's why we trust OpenAI or Anthropic more because of the law of the land, because of the government, because we know that if tomorrow something terrible happens, there's at least they need to answer to the government of the United States. Unfortunately, if it is not in the United States, if it's in some other government, some other country where some other government is there, we are not sure whether they're going to do the same thing or not. And that's why a call which I want to make is I think this is for each and every country, for each and every government in each and every country, they should have those kind of laws of land and have those kind of processes where they can call a company up and ask them why or how the user data is processed.

35:31And that kind of builds a lot of trust. So that's why it's a shared responsibility, unfortunately. It's just not what the company can do. Yeah. Yeah. You did a report recently about the security of retail apps and in the recent holiday season, there's, you know, everyone's buying. I mean, what did you find? What are some of the surprising findings from that? One thing, so the reason why we did the report is very interesting is because there was a Black Friday sale which started happening and I'm like, I need to, you know, buy some gadgets for myself. And then I realized there's so many people who are actually utilizing applications to actually buy other things or even pre-book things in Black Friday.