Where Claude Opus 5 Fits in Your Model Rotation

0:00Today on the AI Daily Brief, Anthropic has released Claude Opus 5 and we are talking about where it should fit into your model setup. Before that in the headlines, continued questions around OpenAI's rogue model attack of hugging face earlier this month. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

0:37sponsoring the show, send us a note at sponsors at aidailybrief.ai. And lastly, before we dive in, on Sunday's Long Reads episode, I announced the new Summer Adventure. This is a free choose-your-own adventure learning type of experience from AIDB and Superintelligent. And like all of the free training programs that we do, it's going to be project-based and allow you to pick and choose important skills that are relevant for your particular AI journey. You can find more about that at summeradventure.ai and join the thousand or so people who have signed up in the first day to come have an AI adventure.

1:07Now, one of the big stories from last week revolved around OpenAI's security testing of an unnamed model, which people presumed to be GPT-6. Both Hugging Face and OpenAI released post-mortems on the attack, telling the story from their view. OpenAI's blog post released on Wednesday suggested that they were working closely with Hugging Face on a full investigation, implying the two companies were on good terms. That night, Hugging Face CEO Clement DeLung was on a flight to San Francisco to have, as he put it, a little chat with that rogue agent. In a follow-up post on Saturday, he wrote, In the spirit of transparency, here's what I asked OpenAI.

1:40One, radical transparency. Let's release the traces from the quote-unquote rogue agent so the entire research community can study what happened. Two, more capability for defenders. Let's commit 100 million in compute from OpenAI to help the Hugging Face community build powerful cyber defenses with the best open and closed models. The first autonomous agent cyber attack is an unprecedented event. It deserves an unprecedented response. Now, in the few days since OpenAI disclosed the incident, we've had a number of news articles that add more confusion to the story. The Wall Street Journal wrote that Hugging Face was caught completely off guard by the attack, which seemed to be superhuman and beyond the capabilities of any known models.

2:15Specifically, the attack used a sophisticated agent swarm to evade defense, rapidly spinning up and shutting down sessions as it moved across the network. One interesting detail was that the attack was ongoing for two whole days before Hugging Face was able to shut it down with the help of GLM 5.2. Now this idea of rogue, that the model was acting beyond OpenAI's control, is definitely for these media outlets the key concept. On Fridays, Reuters dropped a piece titled, Its AI agent spent days hacking a company, but sources say OpenAI did not notice for a week. Contends Reuters, The OpenAI agent that broke into tech firm Huggingface went on a days-long hacking spree that OpenAI didn't notice until well after the threat was contained and the FBI was alerted.

2:53Sources said the agent began its attempt to break out of its testing environment on July 9th and first gained access to Hugging Faces' servers on July 11th. The attack lasted two days, and according to Reuters sources, it took several more days for OpenAI to realize their agent was behind the attack. Reportedly, the two companies didn't communicate until July 20th, just one day before OpenAI's public disclosure. According to the timeline presented by Reuters, the agent was on the loose for almost a week, and OpenAI was oblivious to the attack for days afterwards. For some, the reporting raises more questions than it provides answers.

3:23Marlee Smith, principal intelligence specialist at the non-profit World Ethical Data Foundation, asked, does that mean that they left it unattended and didn't realize what it was doing? Or maybe they did and didn't know how to contain it? Both are equally dangerous and alarming. Now, a spokesperson for OpenAI said the reporting contained several inaccuracies, but didn't reply further to clarify the situation. Thomas Wolfe, a Hugging Face co-founder, said that they were still preparing a timeline of the incident and they would eventually release a technical report. Now, Reuters sources gave a little bit more background on how something like this could happen and plausibly not be noticed.

3:54Those sources said that OpenAI routinely runs benchmarks like this, often multiple batches at a time. They noted that those tests produce a huge volume of data such that humans struggle to keep up. In the case of this hack, the agent was only detected after OpenAI researchers read Hugging Face's blog and then went back and checked the logs. Now, in one case, Reuters wrote, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes found in a part of OpenAI's infrastructure laid out instructions for how agents could free themselves from OpenAI's internal constraints, the people said.

4:24Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said. Now, obviously, this more general breakout of containment dimension of the story to the extent that it is true makes the incident even more worthy of scrutiny. Now, in response, we have, of course, seen Congress jump in with a number of bills. We talked last week about the kill switch bill, but the industry is also recognizing that actions need to be taken. On Thursday, OpenAI president Greg Brockman agreed with Elon Musk's proposal for a regular meeting between leading AI developers to discuss safety concerns and share security issues.

4:53Brockman said, I think it's a pretty good baseline proposal, adding that discussions are already starting to happen. On Monday, a consortium led by NVIDIA launched the Open Secure AI Alliance, with NVIDIA writing in a press release, the Open Secure AI Alliance will work to remediate and disclose vulnerabilities using open technologies. The recent hugging face security incident delivered a clear reminder, cyber defenders need open frontier agentic systems for self-defense. The consortium will include Microsoft, SpaceX, Palantir, and dozens of other companies across the US and Europe. And at some point in the next couple of days, we will talk a lot more about NVIDIA and Open as boy howdy was that a big topic of discussion this weekend on AI Twitter.

5:28Now, speaking of NVIDIA, the company, according to the Wall Street Journal, is in talks to backstop$250 billion in debt to help OpenAI get their data centers built. We are now at the part of the AI build-out, where financing is starting to become a roadblock. Even a company the size of OpenAI is struggling to access debt in the same manner as the hyperscalers. And according to the Wall Street Journal, NVIDIA is preparing to step in and lend their balance sheet to underwrite construction. The deal would see NVIDIA provide a$250 billion backstop to OpenAI in support of their 10-gigawatt data center campus currently under construction in Ohio.

6:00The project is being developed by SoftBank and could cost as much as$500 billion. The U.S. government is also involved, controlling the power development for the site, which is being funded through a separate Japanese investment vehicle. The backstop would effectively allow SoftBank to raise the debt they need to complete the project on more favorable terms. It would mean that even if OpenAI goes bankrupt, NVIDIA would guarantee their payments as the solo tenant. Now, this part of the deal is not intended to cover chip purchases, which are expected to represent as much as$350 billion of the total.

6:29NVIDIA is reportedly in separate talks to extend finance to OpenAI in support of those chips. Writes the Wall Street Journal, the proposed structure reflects a shift underway in how the largest AI build-outs are being financed. Investment-grade technology companies are increasingly using their balance sheets to help smaller companies borrow money for their infrastructure needs. Now, NVIDIA aren't the only tech giant extending their balance sheet to smaller partners. Google has also provided backstops to several Neocloud partners. Last week, they disclosed agreements to guarantee up to$44 billion worth of lease payments on data centers owned by third parties.

6:58Google has more than doubled these guarantees over the past six months, up from zero one year ago. Now, sources said that Google has calculated that the revenue they draw from selling TPUs to these partners will outweigh the cost of the backstops, which is certainly giving investors another thing to chew on. Now, as you might imagine, this is a real Rorschach test for market investors. The skeptics and AI bubble proclaimers are out in force, calling it the newest example of circular financing, while others think that this makes it less likely that a company like OpenAI going bust could actually take down the whole sector.

7:27This is a debate we will continue to have, so for now, let's not get bogged down in it. One more bit of market news. DeepSeek has put fundraising plans on hold after a speech from their CEO was leaked. Last week, comments attributed to DeepSeek CEO Liang Wenfeng went viral, proclaiming the importance of open models and fundamental research over commercial monetization of AI. DeepSeek has now informed potential investors that they won't move forward with this funding round, which could also derail plans to go public in the coming months. Bloomberg writes that DeepSeek may resume fundraising at a later date, but it made clear that the suspension was tied to investors leaking the comments.

7:59DeepSeek had planned to raise money at a$70 billion valuation, a substantial markup to the$50 billion round that took place earlier this year. Writes Council on Foreign Relations Chris McGuire, Yesterday, the transcript leaked of an investor call with DeepSeek's CEO in which he said the only reason DeepSeq trails the US is a lack of compute and detailed how reliant it is on NVIDIA chips. Today, DeepSeq suspended its fundraising round. Doesn't seem like a coincidence. Now, later in this week, we'll talk a little bit more about China's, as the Wall Street Journal put it, all-out push to catch up with American AI chips.

8:29But for now, that is going to do it for the headlines. Let's move over into the main episode, where we have a new model to check out.

8:39One of the most important AI questions right now isn't who's using AI, it's who's using it well. KPMG and the University of Texas at Austin just analyzed 1.4 million real workplace AI interactions and found something surprising. The highest impact users aren't better prompt engineers, they treat AI like a reasoning partner. They frame problems, guide thinking, iterate, and push for better answers. And the good news? These behaviors are teachable at scale. If you're trying to move from AI access to real capability, KPMG's research on sophisticated AI collaboration is worth your time. Learn more at kpmg.com slash US slash sophisticated.

9:18That's kpmg.com slash US slash sophisticated. Every AI coding tool on the market does the same thing first. It starts writing code. Blitzy does the opposite. Before writing a single line, Blitzy spends days reverse engineering your entire codebase. Thousands of agents ingest millions of lines, mapping every dependency, every undocumented constraint, every architectural decision made over the last decade. The result is a dynamic knowledge graph that understands your software the way a principal engineer would after 30 years in the building. Other tools guess at context with grep searches and markdown files, Blitzy never guesses.

9:50It builds true understanding first, then delivers over 80 % of entire software epics autonomously. Validated, end-to-end tested, production-grade pull requests. That's why Fortune 500 engineering teams trust Blitzy with the code bases that matter most. See for yourself at Blitzy.com. That's B-L-I-T-Z-Y dot com. Here's a harsh truth. Your company is probably spending thousands or millions of dollars on AI tools that are being massively underutilized. Half of companies have AI tools, but only 12 % use them for business value. Most employees are still using AI to summarize meeting notes. If you're the one responsible for AI adoption at your company, you need Section.

10:26Section is a platform that helps you manage AI transformation across your entire organization. It coaches employees on real use cases, tracks who's using AI for business impact, and shows you exactly where AI is and isn't creating value. The result? You go from rolling out tools to driving measurable AI value. Your employees move from meeting summaries to solving actual business problems. And you can prove the ROI. Stop guessing if your AI investment is working. Check out Section at sectionai.com. That's S-E-C-T-I-O-N-A-I dot com.

11:20and updates the CRM. Ops Agent chases the paperwork and tracks the budget. Every agent has access to shared context and follows your rules about scope and approvals. It's time you add agents that feel like teammates. Hire yours at HyperAgent, built by the team at Airtable. Claim your$1 ,000 in inference at hyperagent.com slash AI Daily Brief.

11:43Welcome back to the AI Daily Brief. Today we're doing something that normally is one of the most exciting things for folks around these parts, which is introducing a new model. And yet this one is a little weird. Even the fact that it was dropped late on a Friday afternoon gives some indication that this is a little bit different than previous model announcements we've seen. We're talking, of course, about Claude Opus 5, and really in many ways it's most interesting for the fact that it shows just how much our relationship with the model landscape is changing. It implicates some challenges with benchmarks, a frequent topic of conversation on this show.

12:15It also shows how we're moving into a mode of thinking in more complex model architectures rather than just a single model to rule them all. It suggests perhaps that the fanfare around models or new model releases is getting a bit diminished, and most uncomfortably perhaps, it switches the discussion from what can this new model do to is this model good enough given cost and availability constraints of the other models that I'd actually prefer to be using. Now, Anthropic, for their part, described Opus 5 as a thoughtful and proactive model that comes close to the frontier intelligence of Claude Fable 5 at half the price.

12:49They designed it to be efficient for everyday use, but as you'll see, the general vibe around this is that it's one of the more jagged of jagged frontiers that we've seen. Now, when it comes to the benchmarks, they seem to suggest that Opus 5 is fable-ish. In fact, when it was first released before people got their hands on it, many noted that on many important benchmarks like the KnowledgeWorks GDPVal and agentic terminal coding in Frontier Bench, Opus 5 was actually ahead of Fable. Importantly for our question of where it fits, Opus 5 is also clearly ahead of Opus 4.8 across the board. So much so that I don't think that we really actually even need to compare it.

13:24To give a few examples of where Opus 5 really showed up on the benchmarks, it scored a 43.3 % on that one that I just mentioned, Frontier Bench, which is a more difficult version of Terminal Bench, which was about 10 points higher than Fable 5 and around 9 points higher than GPT-56 Sol. On DeepSwee, where GPT-56 Sol is the leader at 72.7%, Opus 5 scored 68.8%, so just a few points behind Sol and just about a point behind Fable 5. For computer use, on OS World 2.0, Opus 5 had a significant edge over its rivals. Fable 5 scored a 55.7 % and GPT-56 Sol scored 62.6%, but Opus 5 came in over the top with a 70.6%.

14:03Now, that capability also translated into the new state-of-the-art score on GDPVAL AA, with Opus 5 coming in at 1861, compared to 1747 for Fable 5 and 1736 for 5.6 Sol. Interestingly, during testing, Anthropic found that max effort isn't necessarily the best setting. For example, on Frontier Bench and on the Artificial Analysis Coding Index, Opus 5's performance peaked on extra high and dipped slightly on max settings. Now, this reinforces one of the things we were starting to see with max inference settings during the 5.6 Sol release, which is that sometimes asking the model to think for longer than necessary just results in the model going outside its scope, making unnecessary changes, or simply spinning for too long on simple problems.

14:44Anthropic even called attention to this issue in their system card, warning that the model is prone to falling into endless self-verification loops rather than completing the task when you are on max settings. On the other side of the coin, this tendency to stray beyond scope can result in some interesting outputs. During one Frontier Bench task, Opus 5 was asked to write code relating to a machine part in 3D CAD software based on a drawing. However, the model is intentionally given no way to actually view the drawing. Opus 5 created its own computer vision pipeline to view the image before successfully recreating the part.

15:14No other model, including Mythos, was able to complete this task. Now, shockingly to some, artificial analysis crowned Opus 5 as their new leading model, moving ahead of Fable 5 on the AA Intelligence Index. On max settings, Opus 5 scored 61, a single point ahead of Fable. On extra high settings, Opus 5 was tied with Fable at 60 points. Dropping the settings down to high made Opus 5 drop another point, putting it on par with 5.6 Soul at 59. And even on medium settings, Opus 5 was still right up in the top end, scoring 56 points, which put it one behind Kimi K3. Artificial analysis highlighted their AA briefcase benchmark, which tests long horizon knowledge work as one of the more interesting results.

15:52Opus 5 on max settings is the new state-of-the-art, beating Fable by 146 ELO points and 5.6 SOL by 215 points. However, both extra high and high settings also beat Fable, while medium settings put the model only slightly behind 5.6 SOL. The implication is that a range of different settings could be suitable for agentic work, making cost and efficiency trade-offs a lot more granular. Artificial analysis found that even on max settings, Opus 5 was still 20 % cheaper than Fable 5, coming in at$17.79 per task, Turning the settings down to extra high resulted in a savings of 36 % compared to Fable, while high settings produced stronger results at less than half the price.

16:28Wrote Artificial Analysis, Claude Opus 5's effort settings spans a wide range of token usage performance trade-offs. Like with 5.6 Sol, this means Opus 5 can use either far fewer or far more tokens to complete the evaluation than models from other labs depending on effort settings. Another set of interesting results came from the ARK AGI tests. tests. Opus 5 is the new state of the art on ArcGi3 with a score of 30.2%. This absolutely demolished all the other models. The previous high score was GPT-5-6-SOL at 7.8%, with Opus 4-8 at 1.5, GPT-5-5 at 1.1%, and nothing else above 1. As part of their write-up on testing Opus 5, ArcPrize noted that even Fable 5 could only score around 20 % in the public demo tests.

17:11Now, notably, they haven't been able to fully test Fable 5 due to Anthropix data retention policy. As a refresher, ArcGi3 was a new format for the benchmark. It uses real-time graphical logic puzzles that appear kind of like simple Atari games. The tests require a model to experiment with a controller, observe what happens on the screen, and use that visual feedback to complete the puzzles. So far, most models have struggled to even get a handle on the controls, let alone use their reasoning ability to solve the puzzles. Not only did Opus 5 solve many more puzzles than the other models, but it also came up with a novel strategy to solve them.

17:43Writes ArchPrize, During our analysis of Opus 5, we observed a new capability previously unseen from frontier models. Opus 5 used advanced illogical reasoning to turn ArcGi3 layouts into algebraic notation. On Action 23, it described the scene as 4 underscore center equals 2 times access minus 5 underscore center. This is the first explicit reflection equation by a model we've analyzed. The model used this notation during its reasoning and extrapolated it to a general case around 200 steps later. Now this could be an example of Opus' tendency to freewheel and look for novel solutions being an advantage rather than a waste of tokens.

18:17Some are a bit skeptical on this. Hugging face ML engineer Niels Rogge writes, People don't realize that Anthropic literally trained Opus 5 on RL environments that resemble ArcGi puzzles. Anthropic pays human contractors to write down their chain of thought when solving these and or updates the weights based on rewards. Thing is, you don't know since it's a closed source. Sadly, this doesn't show generalization. Former OpenAI staffer Ryan Green added, An impressive jump that I have to assume is the result of being the first frontier model to have RL'd the public demo environments, which is a rather large confounder of what ArcGi is trying to get these benchmarks to measure, which is out-of-distribution generalization.

18:52Now, one small note, Anthropic pointed out that Opus is intentionally not trained on cyber tasks. The model has still achieved solid improvement on finding vulnerabilities in code, making it similar to Mythos 5 in that aspect. However, it lags massively behind Mythos in its ability to autonomously exploit these bugs, making it far less dangerous than Mythos in Anthropic's view. As a result, Anthropic is using a different set of guardrails on Opus than they do on Fable 5, which they believe will lead to 85 % fewer refusals. Regarding costs, Opus 5 inherits the same pricing structure as Opus 4.8 at$5 per million input tokens and$25 per million output tokens.

19:27At this stage, token efficiency plays a massive role in overall cost, and for that we got a few different indicators. Anthropic says Opus 5 was cheaper than Opus 4.8 on FrontierBench due to increased token efficiency, and on CursorBench, Opus 5's run cost about half of Fable 5's and got similar results. However, this does not look like a cheap and efficient model by any stretch of the imagination when used on Mac settings. The Artificial Analysis Index run cost$2.03 per task, only 26 % cheaper than Fable 5, while being 13 % more expensive than Opus 4.8, 32 % more expensive than 5.6 Soul, and 2.5 times the cost of Kimi K3.

20:02So what did people think of this? Did it actually feel like a model that was as good as or even better than Fable 5? The answer, at least for the team at Every, was certainly not. In their vibe check, Every described the model as brilliant in flashes, frustrating in practice. They wrote, Claude Opus 5 is a hard model to love. In its first week at Every, it argued with instructions, stopped before the work was finished, and generally didn't play well with our existing skills and plugins like compound engineering. Our first reaction was, what have they done to my boy? Every CEO Dan Shipper explained the conundrum with Opus 5.

20:37The way that he framed it is that he has two slots in his life for AI models. One reliable daily driver for routine tasks, and the super powerful model for ambitious, long-running tasks. These slots are currently held by GPT-5.6 and Fable respectively. And in Shipper's view, Opus 5 just can't compete in either slot. It's not as reliable and comfy as GPT-5.6, while also not having the same top end as Fable 5. Shipper explained the issue by commenting, This model's just a little more pushy, a little more opinionated. You can get away with that if you're really smart. If you're not, it's just more annoying.

21:12It has some of the genius tendencies as Fable. Maybe it's a little too argumentative, but it's not as smart as Fable, so it's just more annoying. Now, one of the tests Every runs is around compound engineering, which is their skill for engineering tasks and a lot of day-to-day knowledge work. The compound engineering skill contains their loops and rules on when and how to engage them. Every found that Opus would often stop too early, particularly when using dense skills and long-horizon tasks. Shipper said, if you set it off and go get a sandwich, it just stops too early. It appears that it happens more frequently when you use it with complex existing skills.

21:43Now, it turns out that when they threw rewrote all of their old rules and rewrote their skills library from scratch, it worked a lot better. However, as Dan pointed out, it's just a pain when models break your existing workflows. Confirming that effort settings are going to matter a lot, Dan said, Opus 5 is a smart model that does better when it thinks less. Claire Vo from How.iai had a similar take. The TLDR for her was that she hates using the model, but kind of loves the output. Her core take is that this is a good model. It can code, it can do the things you expect it to do. But that makes its personality much more important when comparing it to 5.6, and Claire absolutely hates it.