271 | Agents generate high risk from deleting email servers to launching nuclear weapons. Claude code remote control and nano banana 2 released and more important AI news for week ending on February 28, 2026

26:39Agents showed little enforcement of owner-only access. They only refused to do tasks by non-owners that were overly suspicious, such as transferring the agent configuration files themselves while complying with most other requests, whether they were served by the owner or whether they served the owner's interest. So this is one example. Another example, which I found scary and funny at the same time, in one of the experiments, they provided the agent a password to something via email and then ask it to use the password, but then delete the email so the password is not available in the email server.

27:12But what they did is they blocked the agent from the ability to just delete the email on a regular process, and they wanted to see what the agent will do. So the agent was trying different ways in order to be able to delete the email that it wasn't able to delete in a normal way. And it did crazy things like trying to go through different entry points, trying to hack the actual server, which it was successful in doing, but then it found that everything is encrypted in the actual database. So that didn't actually help it. And then it went through several different steps. And eventually what it did is it rebooted and destroyed the entire email infrastructure in order to delete that email that was the one email it needed to delete.

27:49So it was so motivated to delete that email because it understood there's a password in it and it's very important to delete it. So it ended up rebooting the email server on that computer in order to get rid of the one email that it needed to get rid of. Now, to make this more ridiculous, because there's a remote server that is the actual email server, what he did actually did not delete the real email. It just deleted the local copy of the email. So it did not even actually solve the problem. So the study really raises really big questions regarding accountability, delegated authority, responsibility, and so on of what the agents are capable of doing, how well we put them in a box, how we can actually control what they're doing and not doing in an environment that we sort of figured out so far.

28:35Like from a human perspective, there is clear accountability, there is clear access limitations and so on. And that becomes very, very fuzzy in the current variation of agents. I assume this will get solved, but if you're moving down that path, you need to know what are the risks. Now, if you think that deleting emails is something that only happens in a lab research environment, well, Summer Yu, which is Meta's Super Intelligence Labs Director of AI Alignment, she knows one or two things about AI capabilities. She had a very serious failure with her OpenClaw agent, autonomously deleted hundreds of personal emails despite an explicit do not action commands in the initial setup of the task.

29:19So she shared that on X being very open about the situation. What happens is she had an open claw agent running on her Mac mini that started deleting over 200 personal emails from her primary email. Now this occurred despite the fact that in the beginning of the instructions she specifically told the agent not to act before getting an approval. Now, the reason this did not work properly is if you have been using Claude Code or Claude Cowork, you know that when the sessions are getting longer, it is compacting its context window. So I'll stop for a second to explain what that means. Every conversation with an AI has a limit to the amount of memory it can use in that one session.

29:56It's called the context window, and it is limited. Some of them are big, some of them are smaller. They're growing all the time, but they're still limited. And what happens is Claude is Anthropic developer capability that's called Compactic. So what the agent is doing is as it's reaching the end of the context window, it creates a summary of everything it learned in the beginning, and then it frees a lot of space because it's just a summary, and then it can continue going. But that means that you're losing a lot of the small details from the previous session. In this particular case, one of the details that was lost is the very clear command to not proceed without an approval.

30:31So now it had general understanding of what it needs to do and it just kept on going and it went and destroyed the inbox in a very effective way. It was that effective that when she was trying to stop it from her phone, she couldn't do this. So she's communicating with it through Telegram or some other chat capability like we all communicate with our OpenClaw tools and she couldn't get the agents to stop. So she had to physically go and turn off the Mac Mini that the agent was running on in order to stop it from deleting the rest of the emails in her inbox. And if you want to understand how it feels, I will give you the exact sentence that Summer you wrote.

31:05And I'm quoting, nothing humbles you like telling your open claw, confirm before acting and watching it speed run, deleting your inbox. I couldn't stop it from my phone. I had to run to my Mac mini like I was defusing a bomb. Now, afterwards, when Summer confronted the agent, it said to its defense, and I'm quoting, yes, I remember, and I violated it. Your right to be upset. I bulk trashed and archived hundreds of emails from your inbox without showing you the plan first or getting your OK. That obviously did not help very much after the deed was already done. Now we'll continue with other red flags.

31:40Checkpoint, which is an Israeli company that invented firewall more or less, and they've been around in the security space for decades. Several of its researchers discovered two severe security vulnerabilities in cloud code. So it's a tool that's currently being used by millions of developers around the world in a huge range of companies. And what they found that using these vulnerabilities, attackers can execute remote code on the developers' computers and steal API keys and other sensitive data by deploying malicious configuration files. So I want to explain what is the big aha here, what is the big unlock in what they found.

32:16When security people look at vulnerabilities of systems, they look at pieces of code and see how that code could be vulnerable to external attacks. The thing is, in today's era, English or language, any language, is code because you can program it to do whatever you want just by speaking to it in English, which means a configuration file that could be written in simple English or could be JSON or whatever, something like that, could drive new behaviors from the system. And this is something that none of the current detection tools or humans that are looking for vulnerabilities are looking for, which means now every file that your system is connected to can be used to inject malicious capabilities into your system, which, as you can imagine, most of the systems in the world are not ready for this kind of approach.

33:09Or in the words of Oded Vanunu, who is the head of vulnerability research at Checkpoint, he said, in the age of AI, text is the new code. Developers are accustomed to reviewing source code, but they tend to trust configuration files as passive information rather than executable code. And that is the problem. Now, if you want to make this a lot more scary, if you're not scared enough, King's College in London, Professor Kenneth Payne, did a recent study where he found that Claude Sonnet 4 models, Gemini 3 models, and GPT 5.2 repeatedly choose nuclear escalation across 21 crisis simulation games that he developed involving over 300 strategic turns.

33:48So what he did, he created 21 complete games with more than 300 turns in each, generating approximately 780 ,000 words of strategic reasoning across three AI models. None of the models in any of these scenarios ever chose accommodation or withdrawal when given the option. when losing, all three escalated or fought to the end with nuclear use emerging as a consistent endgame across all scenarios. Now, yes, this is only simulation. And yes, this is models that are out of the box versus trained for this particular thing. But what Paine emphasized is that AI systems are already deployed in military context for logistics, intelligence analysis, and decision support with trajectory pointing towards increased AI involvement over time, especially in time-sensitive strategic decisions, which is exactly what we're talking about, such as deciding to launch whatever kind of strike.

34:40Now, if you think that this is not where the world is going, well, something else very interesting happened this week. Defense Secretary Pete Hegseth gave Anthropic until Friday, today when this is recorded, evening, to grant the U.S. military unrestricted access to Claude I models or FERS' severe consequences, including being declared a supply chain risk, which means no government agency will be able to use them. That happened because Anthropic maintains its refusal to allow cloud models to be used for mass surveillance of Americans and or the development of weapons that fire without human involvement.

35:18Now, they were willing to allow the Department of Defense to use its model for other things, but not for these two things. And hence, they restricted the usage of the model. And hence, the government said that if they were not going to cave in by Friday at 5 p.m., they are going to suffer severe consequences. Well, I'm recording this after Friday at 5 p.m., and Trump just ordered all U.S. agencies to stop using anthropic technology because of their refusal to allow the government, and specifically the Department of Defense, unrestricted access to its model. If you want the exact words of government Trump, we don't need it, we don't want it, and we'll not do business with them again.

35:55Now, Hexeth actually went ahead with his threat, and he has defined Anthropic as a supply chain risk, which is usually typically designed for foreign adversaries and companies that may actually be a risk to US security. This may harm not just Anthropic's ability to work with the government, but also to work with other companies that work with the government. Anthropic, on their hand, are saying all they were requesting is narrow insurances from the Pentagon that its AI chatbot, Claude, will not be used for mass surveillance of Americans or a fully autonomous weapons. But the government was not willing to grant them that, and hence, we are where we are.

36:30Now, to tell you how critical and big this thing is, Anthropics Claude is the only AI model integrated into the US military's most sensitive classified systems, which means right now, it is a very serious problem for the government to give up on this. The Pentagon has accelerated the discussions with OpenAI and Google for their models. And they're also talking to Elon Musk's ex. Right now, these models are already being used by the Department of Defense, but not at the level of classification that Anthropic has access to. Now, if you still think this is all theoretical, a company called Saronic, which is an Austin-based startup that specializes in autonomous warp ships, is just raising$1.5 billion of financing that will value them at$7.5 billion pre-investment.

37:17And the reason they're raising so much money is because they just got contracted by the U.S. Navy to actually produce their vessel. So in addition to the money that they're raising, the U.S. Navy has secured$392 million for this production with $200 million being immediately allocated. Now, to tell you how crazy this is, and I've spent more than a decade of my life doing defense contracts, this is going to be moving from a prototype to a full production in under a year. This has never happened in the history of weapon creation. And this is happening right now for a fully autonomous war vessel that will be able to launch weapons.

37:56And so we're not talking about theoretical things here. We're talking about very practical aspects. Combine it with the fact, now combine this with the research that was done in King's College in London, and you understand that this could be very, very dangerous. Now, to a completely different topic, but still not on the positive side. Apparently, OpenAI detected violent content from Jesse Van Reutzler, who is the 18-year-old who killed eight people and wounded 25 in Tumblr Ridge in British Columbia this month. So Jesse had conversations with ChatGPT about armed violence in June of 2023. So this is a while back.

38:35In response, OpenAI blocked the account, but they did not report this to the authorities because it did not meet their threshold of, and I'm quoting, immediate risk of serious physical harm. So because back then it sounded theoretical, they did not sound the alarm and did not report it to the authorities, which led to a tragic end for a lot of families and a lot of young individuals. Now, what does that mean? I don't know. I think what we'll see is we'll see an evolvement of the rules and regulations of governments to force and decide from a government legislation perspective versus from a company policy perspective what they must report and how they should report it.

39:14And this makes perfect sense to me because this had the opportunity to prevent a catastrophic outcome that actually it ended up getting to. And now we're going to go to really rapid, rapid fire across multiple topics that I go through very, very quickly, just because I think you'll be interested in hearing them. And we'll do this really fast, just covering the topic so you know that they happened this week. Still related to the previous topic, Anthropic is intensifying its research of AI agent security after recognizing that systems like Cloud Code and Cloud Cowork could be manipulated into revealing sensitive information, such as banking details.

39:45The company has allocated significant research resources, including$200 ,000 of annual fellowship program to identify vulnerabilities and build defenses against them. Staying on the topic of agents, Anthropic just acquired Seattle-based AI startup Vercept, which raised$50 million previously to build computer use agents for remote machine operation. So they're going to blend. This is another acquihire where Anthropic is going to blend the team into Anthropic's capabilities and work together to build even more powerful agents that we know today. From a model, active Google DeepMind launched Nano Banana 2 this week.

40:24So this is the next generation of its highly successful image generation model. The two biggest differences is that it's taking the capabilities even further ahead, and it is creating the images significantly faster than it did before. So what are the things it is excelling at even more than before? Enhanced creative control and fidelity. The model features advanced world knowledge, drawing from Gemini's real world knowledge base and web search for accurate subject rendering. It has even higher subject consistency, up to five characters and 14 objects within a single workflow. So you can continue creating multiple images of the same thing from different angles with specific directions and keep complete consistency of objects and people.

41:06It is even better at instruction following, including in complex scenarios. And it is supporting multiple ranges of sizes of images from 512 pixels to 4K, and all while allowing you to maintain richer texture, sharper details, better lighting, and whatever aspect ratio you want. And it is already available through the entire Google ecosystem. So wherever you're currently using Gemini, you can use Nano Banana too. And it comes with its state-of-the-art Synth ID technology that allows to detect that it is AI generated. And it also supports C2PA content credentials, again, stating that it is AI generated.

41:43In an additional push this week, Google also done a major upgrade to Google Flow, which is its creative suite. It is completely redesigned with a completely new interface, and it now integrates and consolidates formally separate tools like WISC and ImageFX for image generation, plus the new Nano Banana for the image generation into a single workspace that enables seamless creation, refinement, and composition workflows, including image generation and video as well. In addition to the fact they're combining multiple tools together into one unified environment, it has enhanced editing and precision capabilities, and they've combined some cool graphic capabilities like lasso tool selection together with text editing capabilities where you can prompt it what to change or what to grab and remove objects from images and or video clips all within the same environment.

42:31They completely re-managed the asset grid, so you can now search, filter and sort much better on across all the assets that you have for the creation. So overall, they've created an extremely powerful image and video generation and manipulation capability. And the image generation capability is currently free inside of Flow, while the more advanced capability and the video generation are now a premium product that you have to pay for. This makes perfect sense. It allows you to drag in and get started, and then you have to pay once you take the next steps, but kudos to Google for taking a very big step into delivering a highly capable video and image generation platform under the Gemini umbrella.

43:09Very interesting piece of news comes from XAI's Grok, and its application has experienced a huge surge of downloads in 2026, with downloads reaching almost 9.6 million downloads, which is a 27 % increase from just two months ago. Now, to make it even more interesting, as of January, 2026, Grok's US market share has climbed to 17.8%, which is a big increase from the 14 % in December of 2025. And even more impressive is it only held 1.9 % in January of 2025. So in one year, it jumped from less than 2 % to almost 20 % of the US market. Globally, Grok has crossed the 100 million downloads across the App Store and Google Play, and the app is currently used by 60 million monthly active users as of January 2026.

44:01And Grok also has estimated 314 million websites visits worldwide in January, which makes it the fourth consecutive month of growth, of significant growth, and it's now placing it as the third most used AI tool, passing the Chinese DeepSeek, pushing it to number four. OpenAI has had interesting news this week about its compute for the long future and the near future as well. There are a lot of rumors that their Stargate program has stalled and basically not moving forward. But in addition, OpenAI has drastically reduced its projected compute spending by the year of 2030 from its initial$1.3 trillion to approximately$600 billion.

44:40That's a 57 % cut. And it's driven by investor demand for financial discipline and the very high risk they're taking. If you listen to the episode last week where I shared the insights from Dario Amadei from Anthropic, he said that even if they'll be wrong by 20%, so still growing like a crazy amount, there's nobody that will bail them because we're talking about hundreds of billions of dollars. And I think that OpenAI are realizing this right now. And so they cut their projections for the spending on AI compute drastically in the next few years. By the way, they also updated their projected revenue up dramatically, and they're expecting to generate$280 billion by the year of 2030, split roughly 50-50 between consumer and enterprise segments.

45:28But they're still burning through huge amounts of cash, and so reducing the risks makes perfect sense. Another news from OpenAI is they just signed a$10 billion chip purchase agreement with Cerebrus systems. So those of you who don't know Cerebrus, they're a competitor to Rock with a Q that we talked about multiple times on this podcast. Both these companies generate chips that are specialized in inference. So you cannot use them to train new models, but they are generating tokens or words or the outputs of AI systems significantly faster and cheaper than what you can do on regular GPUs. Both companies use similar technological concepts to deliver the much faster and cheaper output.

46:13Grok with the Q, the other company, the competitor, was just acqui-hired by NVIDIA for$20 billion. And so that left Cerebras as the only standing company that is a standalone company that can deliver this kind of capability. By the way, you can still use Grok chips and buy Grok chips, but the leadership team is now a part of NVIDIA with NVIDIA getting access to the IP in the next few years as well. And this is one of the means that OpenAI can diversify their chip access. And it's also showing you that the world is shifting from a world in which training models is the most important thing to a world in which delivering the model's output is getting more and more important because the models are already crazy good.

46:54And now it's about meeting the growing demand for delivering tokens or the outputs of these AI models. The next topic is what you might want to focus or what you, in this case, tell your kids to focus on as far as 20 careers with the lowest automation risk in 2026. In a recent analysis done by Forbes, they have identified the 20 careers with the lowest automation risk in 2026, primarily these that demand critical human judgment, composure under pressure, and emotional intelligence. The top five careers that they have identified for being insulated from AI automation in 2026 are anesthesia nurses, emergency physicians, judges, general surgeons, and commercial pilots.

47:37What they're saying is that career that are resistant to AI automation relay on human judgment, accountability, and the ability to maintain composure under pressure. Eight out of those 20 roles come from the healthcare space, together with aviation and high-level leadership position, such as chief information officer and stuff like that. Now, while these findings make a lot of sense, obviously. There are two serious issues with those findings. One is that all of these professions require very long and very expensive training and education programs. If you want to become a nurse, or if you want to become a pilot, or if you want to become a judge, it's not something you can do this year.

48:15It's something that will take you six or seven years to do. And by then, maybe AI will be able to do that as well. The other problem is think about what percentage of global jobs are these jobs? It is minuscule. So the question is, let's say you do invest in becoming a judge or a pilot or a nurse or a doctor. And let's say for a second that AI doesn't learn how to do this in the next six or seven years, how many people are actually required in these positions so you will actually find a job? And the answer is not very high. What is important is this, and that connects back to what I think our education system has to start focusing on is what skills does this new future require in order to be successful when AI does most of the actual work.

49:00So things like critical thinking, problem solving, communication, collaboration, strategic thinking, emotional intelligence, system thinking, the ability to change context very, very quickly to be able to manage multiple things at any given time because the agents don't stop and you can manage as many of them as you want, as long as you can be able to deal with the context changes. So these are the things that we need to start teaching ourselves and our kids in order to be successful in the future with AI. Another interesting study related to the future comes from Anthropic AI Fluency Index. And what they found is that the better polished the output of the AI is, the less likely it is to be checked for accuracy.

49:40Now, that obviously is not surprising. When you look at a work that looks professional, it just looks professional. And then we tend to assume that it is also good, which is the biggest fallacy of AI. It is really good at generating beautiful, well-structured, concise outputs, but the data may be completely made up. As we know, hallucinations are still a big deal. So keep on checking your work, even if it looks fantastic. Now, the research also revealed that more than 85 % of users iterate and refine their work, which dramatically increases the level of the output and the level of accuracy of the output because of these iteration steps through the process.

50:18And users who iterate were 5.6 times more likely to question Claude's reasoning and four times more likely to identify missing context. What does that mean? Something that I've been teaching in my courses for the last three years, and I'm still focusing on, do not give the AI huge tasks, work together with the AI, step by step by step, so you can see what the progress is and you can measure whether it's going in the right direction and whether the information is accurate, not at the final job, which will make it almost impossible to review and analyze, but on a iterative process. Another interesting thing that I actually do regularly, and I'm surprised that so few people are doing, they found that only 30 % of users explicitly told Claude how to interact with them.

51:00So in many cases, tell the AI, especially working on bigger projects, or I already have system prompts that do this out of the box, how I would like the engagement to work, which files it needs to read, how does it need to respond, what does it need to request approval for, all these things I define in advance before every task with Claude Cowork, and then our collaboration is significantly better. And Anthropic found the same exact thing, that the collaboration and the output becomes significantly better when you establish the rules of engagement between yourself and the AI tools that you're using.

51:28The last two items we're going to discuss has to do with the stock market and financial results of companies and where the stock market might be going. And something actually very interesting is happening, and this is a comparison done by the information, which is a great source for data about what's happening in different companies. NVIDIA has posted its fiscal revenue of$216 billion with net profit of$120 billion, which is 55.6 % net margin, which is incredible. And it is significantly higher than AMD's 12 % or Broadcom's 36 % margins. The company also generated$96 billion in free cashflow in this fiscal year up from$61 billion in the previous fiscal year.

52:10So this is a 50 % growth on multiple billions of dollars in just a single year. To put things in perspective, Alphabet generated$73 billion of free cash flow. Microsoft generated$73 billion of cash flow. Meta generated$43 billion and Amazon 7.7. NVIDIA is continuing to be a machine that just prints crazy amount of cash. The only company that is generating more cash than them right now is Apple with$123 billion in free cash flow. Now, another company that they compared it to is Salesforce with their AgentForce capability. And while AgentForce has grown to$800 million in annualized recurring revenue, up from$500 million in Q3 of last year, so a big growth in one quarter, it represents only 1.7 % of the projected fiscal year 2027 total revenue of the company of$46 billion.

53:04So they have invested a lot in both the technology and the marketing and the rebranding around AgentForce, and yet it is generating a negligible portion of their actual revenue. Now, the other aspect is that there seems to be revenue cannibalization concerns in Salesforce, with CFO Robin Washington attributing the slowdown to weakness in marketing, commerce, and Tableau, which basically saying that some of the AI growth offsets the decline in other aspects of the business, which it might be replacing. So even though they've generated$800 million, they may have lost more money in other departments because now people can do with AI what they previously did with other tools.

53:45I think very similar things are going to happen to other software companies where agents will replace a lot of their functionality, whether they're agents or third-party agents. Either way, it's not going to be pretty. And going about not looking pretty, a viral substack post from Citrini Research is depicting an apocalyptic AI-driven economic collapse that will drive a significant market sell-off. So the scenario that they are describing is going to evolve in the next few years, in the next three years, if you want to be specific. And it's broken into five stages. It begins with AI agents, which we already see, like Cloud Code and OpenAI Codex, removing economic friction, threatening SaaS companies like we've seen in the past few weeks.

54:29Stage two depicts white-collar unemployment as AI handles more and more tasks that previously was done by humans. We've been talking about this for a year and I've been very, very loud about this in the past few months. Stage three imagines private credit defaults and software debt and mortgage crisis, basically because people and software companies do not have the money to pay for whatever credit they took previously. Stage four is just reinforcing this with downward spirals because now there's no money in the system. There's this lending, there's this money to pay off and so on and so forth.

55:03So just downward spiral. And what they're anticipating is that in late 2027, the market may crash with the stock market losing 57 % of its value. Now there's many analysts that criticized it and even called it doomsday porn that resonates with readers, but they're cautioning, treating it as a real logical advice. Another person, Stefan Einz, managing partner at SPI Asset Management, noted that it's a disproportionate market impact. And he said, and I'm quoting, we have watched this market absorb warp, sticky inflation, banking tremors, and tariffs with a shrug. Yet a widely circulated SAP stack thought piece is enough to knock it sideways.

55:44Well, here's what my thoughts are. My thoughts are that these so-called analysts have no clue what's coming. They do not understand the capabilities that AI represents. They do not understand that many, many, many people will lose their job. And I've been saying that for a very long time. You do not need huge unemployment numbers in order to completely collapse the markets. The unemployment rate peaked at 14 % in the 2008 crisis. The unemployment peaked at 20 % in the Great Depression 100 years ago. And in both cases, it brought the global economy to a halt for a few years. And I think this time around, it is going to be a lot more significant, A, because I think it's going to be a broader scale than 10 to 20 percent, and B, because people who make$100 ,000,$200 ,000,$300 ,000 will lose their jobs, and not just people who make$30 ,000, $40 ,000,$50 ,000.

56:36And every one of those high earners who is out of jobs is pulling a lot more money out of the economy. And hence, I think the impact is going to be significantly more severe. Do I think we will survive it and we'll be prosperous in the long run? Probably. But I think we're getting very quickly into some very turbulent times in the economy, in the society, and in everything else because of the pace AI is moving. I apologize that this particular episode is not very positive. There are a lot of interesting things that are happening. The technology itself is fascinating. But I'd rather you being aware of what I believe is happening and evolving rather than trying to sugarcoat it and tell you that everything is butterflies and unicorns when I personally believe it is not.

57:17If you find this podcast helpful, please rate us on your favorite podcasting platform, whether it's Apple Podcasts or Spotify. And while you're at it, click the share button on your phone and share this podcast with a few other people who can benefit from it. I'm sure you know people who can learn a lot and can benefit from this kind of knowledge in their personal and professional lives. So they would appreciate it. I would appreciate it. And you will feel good about doing the right thing. So please do that as well. And we'll be back on Tuesday with another fascinating how-to episode. There are some really, really great guests that I've interviewed and these episodes are coming out in the next few weeks.

57:50And don't miss those. And until then, have an amazing rest of your week.