Very interesting to see such a focus on cost for performance here

What really impress me is opus 5 is better in alignment than fable 5!

This is useful to me since I delegate most coding tasks to Opus and use Fable for planning.

it&#x27;s nice to know how to work the thing that fable fails down to when it dislikes your prompt.

Aren’t they planning to remove it even from max and keep it only credit based? OpenAI will be happy if that would happen

Fable 5 is NOT included in Claude Pro subscription

Really? It&#x27;s better than Opus 4.8, that&#x27;s the point.When they release new versions of Sonnet, no-one expects them to be better than Opus.

Why are we still talking like ai is majorly used for increasing shareholder value only? Its coding performance is top notch and quality is increasing at a rapid pace. It wasn&#x27;t even half this good a year back. It even is useful for a subset of math problems.

The illusion of progress and advancement, to appease shareholders, and slightly postpone the looming bubble pop.

If Fable gets correct answer quicker, then you might pay less than doing back and forth with Opus, plus you lose more of your own time.I see no reason for using less able models in my workflows. There is this saying, penny wise and pound foolish

To have an answer to &quot;Sol&quot; GPT 5.6 which is far more cost effective and available than Fable.

This is confusing to me because in their blogpost they show model benchmarks and it spanks Fable pretty soundly in most tests.

You are being downvoted for a fair question and others are extremely wrong and confident.The point is that Opus 5 is the best they can do without needing classifiers and absurdly broad safeguards.

&gt; Claude Opus 5 is not more capable overall than our most capable general-access model, Claude Fable 5Ok then so what&#x27;s the point?

Very impressive headline benchmark numbers. I expected a step change, but not past Fable. That said - it all depends on whether the classifiers make the model unusable...

There are trillions of dollars at play, so probably

It does make me wonder if these firms, some or all, are saving some announcements to coincide with others that hit venues like HN. Companies like Nvidia surely aren&#x27;t waiting, but OpenAI and Anthropic have unusual timing.

Interesting timing to release this on the same day Jensen makes a statement on open source AI.

Thanks for that, looks really good. I can see why they were constantly pushing back fable going out of the max sub with these benchmarks

I wonder why FrontierCodev1.1&#x27;s data lists Opus 5 as better than Fable 5.

Best can describe multiple things.Almost as good for half the cost is something I&#x27;m very comfortable describing that way.

I mean that certainly makes it best-in-class

At half the price and less likely to auto-downgrade, it sounds like a reasonable claim.

Using the most expensive model for all of your agentic coding work hasn’t been good practice for a long time. Not unless you have infinite money to spend.Fable is typically used for key planning, architecting, and review tasks.I think this is a case where you don’t understand the use case, not that the marketing department is making mistakes.

The blog posts figure cites Frontier-Bench for its agentic coding score, and shows Opus 5 beating Fable 5 43.3% to 33.7%.

I think you&#x27;re being overly cynical here. First, I don&#x27;t see any claim that is the world&#x27;s best model for agentic coding. Second, it is absolutely the best model in terms of coding performance vs. dollar, and it&#x27;s raw performance seems very close to the frontier.

Which numbers are you seeing? It does show that it&#x27;s better than Fable 5 in most things related to coding?

In my opinion, the frontier is passed what is really needed for coding. Fable is good as a supervisor.

Yeah I spotted this immediately too. I&#x27;m sorry. You&#x27;re supposed to be a multi billion dollar company and you can&#x27;t even highlight your chart honestly?

Recent releases have said something to the effect (paraphrasing here):&quot;Use &lt;less expensive or older model&gt; for everyday tasks and &lt;other non-critical stuff&gt;. Use &lt;more expensive or recent model&gt; for complex coding tasks, refactoring large code bases, etc.&quot;.Then, the next model&#x2F;release emerges and the previous &quot;best for complex&quot; gets demoted to &quot;everyday&quot;.Obviously, it&#x27;s all relative. But, it does beg the question: was the previous model really good for complex coding tasks or no? I mean, how is it now suddenly only good for the &quot;easy&quot; stuff?

Also it scored worse on DeepSWE than chatgpt 5.6 sol

I like how they highlighted Opus 5 as the best for “Agentic Coding” even though the number is slightly lower than Fable.  Close enough for marketing, I guess!

this feels like the perfect example of an LLM producing a long text document. And end users just using an LLM to summarize it without actually reading it

<a href="https:&#x2F;&#x2F;www.anthropic.com&#x2F;news&#x2F;claude-opus-5" rel="nofollow">https:&#x2F;&#x2F;www.anthropic.com&#x2F;news&#x2F;claude-opus-5</a> - A blog post for those not wanting to go through a 190ish page pdf

so almost fable 5 with 50% cheaper cost? sign me up

It depends on your quality bar. At a fixed level of quality, given a high reasoning sonnet vs a low reasoning opus, the low reasoning opus tends to be pareto optimal.It&#x27;s only when you need even lower levels of cost than opus at zero to low reasoning when sonnet starts to make sense at all.

My understanding is that Opus should be used for planning, macro-level conversations and Sonnet for execution.So, for coding, for example: Opus for solution design and architectural blueprint and then Sonnet for actual implementation.Works out cheaper with minimal loss of quality.At least that&#x27;s my personal understanding and anecdotal experience.

I daily drive Sonnet 5&#x2F;medium because it gets most things right most of the time at first try, while costing a lot less than Fable.Opus can give better results on architectural&#x2F;concept tasks and I use it sparingly, but it still costs more than Sonnet 5. Opus 5 seems to achieve results very close to Fable 5 while costing less (keeps Opus 4.8 pricing IIUC), but still more than Sonnet 5 then.

Rather interesting that this makes sonnet 5 look even worse! There is no reason to use sonnet over opus with low or no reasoning at all.

Let&#x27;s be honest - they&#x27;re also still hiring software devs.  AI still requires skilled humans in the loop and that&#x27;s not going away.

i know the dream for capitalists is to be able to point an llm at something and say &quot;do and&#x2F;or fix it&quot; but we still can&#x27;t even get them to not go quite literally insane if allowed to run for an extended period of timeand you can only kill weyoun, awaken the next vorta clone and have him &#x27;catch up&#x27; on all that its missed so many times before they just end up with a complete mess, so. uh. yeah.doubt they can just &quot;fix&quot; their problems like that.

How many companies at the size of Anthropic can serve the amount of traffic and manage the amount of compute they have?

This is just like any extreme engineering domain. I am okay with occasional delays in Flights, as long as it takes me from X to Y in 10hrs vs months.

Funny that a company selling an AI software developer can&#x27;t use it to fix their infra.Fixing those issues still requires humans.

Fixing such issues requires software and site reliability engineering, of which coding is just a part.The first page of the score card mentions that this model is not capable to replace engineers.

The code it outputs, yes! It&#x27;s fantastic. It&#x27;s just so frustrating that the product and UX before the code output is so bad. Greatness is so close within their reach, if only they invested in product and QA people.

I found the same thing funny with Computer Use from OpenAI. It struggled to open and close Spotify.So dangerous! I can&#x27;t believe they let the public use this technology! &#x2F;s

&gt; Constant bugs, dropped sessions, issues switching models, http errors. It&#x27;s becoming ridiculousAnd memory leaks.

No idea but we have few Team Premium seats and everyone is encountering issues daily for past two weeks. From straight up outages to vscode extension&#x2F;CLI refusing to process messages. It&#x27;s been unusable for most of our work hours past two days.

I’m using both side by side, my employer pays by the token and I have a Max 20x plan.They start hitting timeouts or API errors at the same time on two different computers. As far as I can tell it’s the exact same infrastructure.

I’m on the biggest max plan. It is riddled with annoying bugs for me, only been using it for a little over a month. Settings screen flashes randomly. But most annoyingly: sometimes when forking chats or sometimes for no reason, the UI just straight up eats my previous messages. The model is still aware of them and can recount them if I ask but the visible history is gone. And that’s not even all of them. Fable 5 is just too good that I put up with it but it seriously raises concerns for me that even with infinite compute these companies can’t even deliver a functional chat UI.

I think you’re just lucky. Look at the Claude status page to see just how often they have outages (it’s almost daily). Even most of the green days have issues if you hover over them, they just don’t count them as outages.

I commend you for burning 6 figure losses into Anthropic thanks to their subsidies singlehandedly but I am curious to learn what you&#x27;ve built with this so far, not to be snarky, I just wonder what people actually produce while having these run nonstop.

Over the last 1.5 years, they had a few failures with their auth system; two or three times auth failed for a few hours so I could not work. But otherwise they have been just fine. Some issues, but not significant.

Is it some Claude Team&#x2F;Enterprise only problematic? I&#x27;m using two 20x Max accounts almost non-stop (Fable&#x2F;Opus) for 1.5 years at this point, zero issues with both client and infra sides (from US and in travels). When I&#x27;m reading such messages it feels like either I&#x27;m lucky or it&#x27;s a part of some campaign.

I applied to their reliability team but never heard back.  I would love to help them solve this problem!

Great that there&#x27;s a new model but they could fix their existing infra. We&#x27;re considering dropping our Claude Team sub cause it&#x27;s unusable recently. Constant bugs, dropped sessions, issues switching models, http errors. It&#x27;s becoming ridiculous

I unironically hope Haiku gets an update considering it came out in October of last year and it seems like Anthropic just kind of forgot about it.

Excited to use it? Will we be seeing Haiku 5 next? &#x2F;s

Models benchmarks start to get saturated again!

I do want to add, that I am pretty bummed if Opus 5 is going to refuse the tasks I have been using Opus 4.8 for (neuroimaging). Fable absolutely refuses anything close to toughing neuroscience.

Because they are a private company and get to do what they want.

I found the biggest problem with fable is the random reasoning_extraction refusals as well as cyber refusals when it sees hex because only hackers use hex.

5.6-Sol is a lot more permissive than Opus&#x2F;Fable even w&#x2F; CVP (once you sign your soul away to Palantir via Persona, anyway), while maintaining better capabilities

Because their model previously got blocked by the government for this and they don&#x27;t want a repeat?

I&#x27;ve seen in some random comments an URL thrown around, supposedly allowing to subscribe to their biosecurity programme.Apparently this is the way - if you know, you know :)

So I guess the new opus will not run on my drug discovery project. It&#x27;s just a binary classifier to screen for new malaria drugs. Fable completely have up on that codebase citing bio security concerns. Seems like this domain will go unsupported by Antropic

Because their fear-based marketing gave the US gov justification in blocking them for a while. They wisely didn&#x27;t do that for Opus 5.

This is good news. I&#x27;m doing a lot of work that Fable thinks is AI related right now (it is nothing remotely competitive to Anthropic - it&#x27;s just relatively basic stuff I&#x27;m doing with learning models and so forth), yet it blocks me almost every time. I have switch to GPT-5.6-Sol for this work since it&#x27;s stronger than Opus 4.8.

Anecdotal, but I tried running a few identical biology questions through both Fable and Opus and the classifier was only rejected my queries with Fable.

&gt; Opus 5’s safeguards match those of Claude Fable 5’s, with one change: it now permits source-code vulnerability discovery at all access levels. This means that the model can support defensive cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensivelyWhy can&#x27;t they also allow Fable to do so also? Why is source-code vulnerability discovery limited to a lower capability model? If Fable and Opus have the same safeguards, except for this one change, I see no reason they can&#x27;t also allow this for Fable.

<a href="https:&#x2F;&#x2F;deepswe.datacurve.ai&#x2F;">https:&#x2F;&#x2F;deepswe.datacurve.ai&#x2F;</a><a href="https:&#x2F;&#x2F;artificialanalysis.ai&#x2F;" rel="nofollow">https:&#x2F;&#x2F;artificialanalysis.ai&#x2F;</a>

&gt; Noticed none of the comparisons mention Kimi K3.That&#x27;s by design. Anthropic wants to make open-weight models illegal (not my speculation -- Dario explicitly said so), so I assume they don&#x27;t want to give them any undue attention.

Noticed none of the comparisons mention Kimi K3. Is there a comparison chart?

&quot;Bro, AI model releases have officially overtaken iPhone releases. At this rate, we’ll be getting &#x27;Claude 9.0 Extra Crunch&#x27; by next Tuesday.&quot;

Same cost as 4.8 but better that 4.8. Happy to get more efficient model. 
But is there any reason all companies are releasing models back to back after GLM 5.2.

eager to see how it benchmarks on <a href="https:&#x2F;&#x2F;deepswe.datacurve.ai&#x2F;">https:&#x2F;&#x2F;deepswe.datacurve.ai&#x2F;</a>

yes. At the bottom of the release post it says that they are releasing two new features, one of which is customizing fallback behavior instead of blocking for restricted models

If it switches mid conversation, this is a massive increase in token consumption because it has to re-read your conversation into cache, right?

Presumably it drops back to 4.8 in those cases so it&#x27;s not really worse

Reversing for the most part, though lately I’ve been doing some code obfuscation&#x2F;binary rewriting stuff. Fable will switch to Opus instantly on these and I’m unsure how this will perform. I suppose the only way to find out is to test.

I don&#x27;t think there&#x27;s been any official confirmation, but even Fable safeguards seem to have gotten quite a bit of tuning, is less trigger-happy and less like regex matching.

&gt; Opus 5’s safeguards match
those of Claude Fable 5’s, with one change: it now permits source-code vulnerability
discovery at all access levels. This means that the model can support defensive
cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively.Okay so it’s worse than Opus 4.8 for my purposes I guess?

Surely that&#x27;s tunable? OpenAI lets you tune response characteristics.

I guess character? Fable is more friendly and curious while opus is a bit more deliberate and conservative.

An Anthropic &quot;leak&quot; back in March said that &quot;&#x27;Capybara&#x27; is a new name for a new tier of model: larger and more intelligent than our Opus models — which were, until now, our most powerful&quot;. A second version of the leak had it referring to Claude Mythos rather than Capybara.I don&#x27;t know how systematic Anthropic are about their versioning - I&#x27;d have guessed that major version number increases (4.x -&gt; 5.x) reflect different base models (different pre-training runs), in which case Opus 5 would be a distilled version of the Fable 5 base model (but without the cyber exploit post-training), rather than being Opus 4.8 with additional post-training, but who knows? I don&#x27;t believe Anthropic have said anything about this.

Fable 5 is assumed to be a larger model.It seems plausible to me that RL improvements allowed Anthropic to improve on Opus 4.8, similar to how OpenAI substantially improved upon GPT 5.5 with 5.6 Sol.Fable 5.1 and GPT-6 are rumored to launch in August, presumably bringing those improvements to the larger models.

Benchmarks don&#x27;t reflect the difference between Opus and Fable; you need to talk to them, and eventually you&#x27;ll be able to tell which one is which without looking.I think the best proxy for this feeling is the Artificial Analysis&#x27; omniscience index. Fable has a 40 score, and Opus (4.8) has 27.

Opus is cheaper than Fable. They could probably replace Fable with Opus but why? They would be churning customers to different models for no reason. Even if a model scores better on benchmarks it can always regress in your specific use case, and customers don&#x27;t like that. Customers want to be able to continue using their current model until they decide to upgrade themselves.

Fable has more parameters. In practice it&#x27;s not yet clear which one would be better for different usecases yet but they are more different than one being strictly better.

Fable is better for some really hard tasks, the same way it&#x27;s better than GPT 5.6 Sol, because it&#x27;s a bigger model.

I am very confused about what the difference between Opus 5 and Fable 5 is now. What is the purpose of having two models that are so similar? The main differences I see are cost and marginal capability, according to the Anthropic-provided benchmarks.

It wasn’t long ago at all that the chief problem was “can AI even help me with this” (cost be damned). Until a time when the answer is an unmitigated “yes, obviously”, the frontier labs have everything to lose and nothing to gain on routing, because if they screw it up you might incorrectly decide “nope, it can’t help yet” due to a poor routing decision.

Is there really that much money in bleeding edge scientific research though?This is what terrifies me about this whole ordeal economically. Maybe we get AGI and it is not worth anything close to what we thought it was for those who have a bet on it.I think of what was the direct, economic value in the betting sense of quantum mechanics or relativity? Huge value at the systems level of society but as you scale down towards the individual the value is more and more dispersed to the point I would think any pool of bets would have all not paid off.You can&#x27;t monopolize and commoditize relativity.I almost think there is a kind of dutch book against the AI equity holder because even in the best case scenario the bet doesn&#x27;t pay off anything close to what is expected for an individual bet.

Because they&#x27;re trying very hard not to understand it.Otherwise the expensive-yet-powerful model probably won&#x27;t see much revenue. How much money is there in bleeding edge scientific research? There&#x27;s a lot, but there&#x27;s even more existing capital in paying people people to do college level paperwork, and the bulk of those traffic gets routed to the cheapest model.You mostly don&#x27;t need super powerful AGI to replace the paper pushers, but the frontier labs are trying to position themselves as being uniquely capable of producing super powerful AGI, and also be the ones replacing office workers.Not sure how it will work out for them, but I think model routing is going to poke holes in that narrative. That&#x27;s why I think they&#x27;re trying very hard not to understand model routing exists.

OpenRouter is in acquisition talks with Stripe, fyiI would expect routers to commodify like tokens.

Openrouter should ideally kill in this space and make their model agnostic infra like memory, harnesses, chat applications.

because &quot;less&quot; can be so much faster?

Not everyone is you. Other people probably have a range of tasks that can accomplished with different models.Certainly if I&#x27;m confident that I&#x27;m going to get what I need from a faster model, that&#x27;s what I want to use, rather than wasting time grinding away for the sake of saying of the same answer came from a SOTA model.Given that every chatbot does offer a range of models, it seems clear people do choose among options.

It&#x27;s very common to use a lesser model for a lesser task, resulting in same quality output.  End result: save money while being faster.  In many cases, it&#x27;s a pure win-win.

I generally agree.  Perhaps there&#x27;s only a 5% chance that it would write better code or find a bug that it wouldn&#x27;t have with a lesser model, but the economics of bugs is strong enough that preventing a single bug is worth hundreds of dollars.

I&#x27;m not a customer of those routing systems, but I quite often use different Claude models for different tasks. While most tasks were Opus 4.8, I often used 4.8 to make a plan, prompts, and package kit to setup Fable for a bigger project, then run it on Fable. Or, for broad single-task searches Sonnet with or without &quot;Research []&quot; turned on seemed to work best both faster, lower overhead, and less verbose answers (when I didn&#x27;t want it).OFC, YMMV

And you do not have flat priced quota for Fable 5, right? Because nobody has, as far as I know. So you&#x27;ll probably not route any task to the &quot;current best available SOTA&quot;.Also: quota. Implies you do not have unlimited access even for flat prices. Which in turn implies that as soon as you hit the quota on the most expensive flat price plan, even you will suddenly discover the magic of economically sensible behavior.

There are two types of users: those who are able to use subsidized rates, and those who need to use API rates due to audit requirements, enterprise billing, etc.Model routing for subsidized users takes the form of a &quot;use Opus 5 subagents for implementation&quot; type of system prompt. You lean into a single provider, build tooling around that, and your savings are far beyond anything multi-provider routing can get you.Model routing for enterprises is far more complex - approaches like <a href="https:&#x2F;&#x2F;fireworks.ai&#x2F;blog&#x2F;kimik3-fable" rel="nofollow">https:&#x2F;&#x2F;fireworks.ai&#x2F;blog&#x2F;kimik3-fable</a> become necessary for cost control.

I’m assuming you don’t pay per API call. Every mid-large sized business in the world does.

&gt; For me, anything other than current best available SOTA for any task is unacceptable.Then you must route. An article with lots of upvotes yesterday or two days ago showed that K3+Fable 5 was more SOTA than either of those.

That&#x27;s nice and all, as long as flat-priced quota continues to exist. Seems unlikely to go on forever!

Sometimes you can get an equivalent result in a fraction of the time using a less capable model.The problem is of course knowing ahead of time that the faster model can give you the same result.

Who are the customers though? Honest question, I&#x27;d like to understand it.For me, anything other than current best available SOTA for any task is unacceptable. The only routing rule I need is &quot;the most powerful model I still have flat-priced quota available for&quot;. I mean, why settle for less?

This doesn’t seem obviously true, eg an Anthropic model will never route to Kimi even if it were best suited for a particular task.

Model routing by the model itself requires the model to pull in a lot of context and it&#x27;s likely more efficiently just done by people with the context already in their head, even assuming the model is perfect at routing (which last I checked, Claude definitely isn&#x27;t). I wouldn&#x27;t trust ML model routers.

i think you&#x27;re thinking of subagent routingmodel routing in this case is cross-providerImo the main issue behind model routing is you need to figure out how much intelligence a new task takes, which is a very non trivial problem. Presumably, a organization knows this about their own tasks and is better suited to built in-house compared to outsourcing to a vendor.

This is actually simpler to implement. You can have a AGENTS.md describing who is boot at what and then you have the agents converse over tmux.

&gt; Model Routing will always be done better by models themselves.&gt; The models themselves will get better at this and frontier companies will simply give that capabilityI would never trust something like model routing to the same company that would profit from it, and that goes for telling models doing their own routing when that could easily be trained into the model to make things more expensive. Sort of a conflict of interest.Models are first and foremost trained by corporations.

Model Routing will always be done better by models themselves. 
Plus routing loses context making it more expensive and less reliable.Model Routing is just Bitter lesson. The models themselves will get better at this and frontier companies will simply give that capability

you trust the service provider but not the router?weird, but ok

what&#x27;s the threshold for model routing where you&#x27;re willing to trust the router?For coding my own work I don&#x27;t trust the model router, and it would have to be shown to be to save a real dollar amount.From a buying perspective it&#x27;s a hard sell to save x but lose out on bugs you are probably introducing at an unquantifiable severity and frequency. How much is it worth to hedge your bets by doing every single inference request on the frontier model?How much will it cost to go back later and fix things, but also the meta question of how to be able to decide on a hypothetical unknowable? (You&#x27;ll never know how much better or worse your code was gonna be, it&#x27;s untestable at a project level)

And it&#x27;s not just that model routing is much cheaper: no longer than yesterday we got a post showing that routing between K3 and Fable 5 was more SOTA than either of those.If that is true, model routing is here to stay.It also seems to validate the minimalist approach of pi.dev, where sub-agents from the same company is not the preferred approach (pi.dev believes in neither sub-agents all from the same company nor MCP even you can do it if you want for pi.dev&#x27;s philosophy is to do add any functionality you want to a minimal harness).Now of course we&#x27;ll get for a few weeks all the Anthropic fanbois and shills explaining that &quot;sure, K3 was basically at the level of Fable 5 but now that Opus 5 is out, open-weights models are six months behind&quot;.

lol I had to get ChatGpt to explain to me the difference between 5.6 sol, 5.6 Terra, 5.6 Luna, 5.5, 5.4 mini, 5.3 spark, and then there is low, medium, high, extra high, max, ultra, and pro… I still don’t really know, it feels like ordering hot wings.

I am not convinced this will end up being a domain of the ‘routers’ vs the clients, as in harnesses themselves. Thoughts?

How does model routing work if the prompt is static? Do the routers attempt to tweak the prompt to make it the best version for that model?  For example, reading the “migration” guides for Claude Opus 4.6 -&gt; Opus 5 is exhausting. Adding in all of the other models complicates it even more. Especially across  vendors.

Looking at all these releases it’s not a surprise that model routing is the fastest growing segment in AI right now.There are 10+ LLM companies, each with dozens of models of different modalities, each model with multiple size variants, then different “thinking” levels, then agentic modes, “pro” modes, a “fast” option, standard vs flex vs batch execution. And of course each end combination has a different input&#x2F;output&#x2F;cache token price.Companies that say “give me a prompt and I’ll route it to the most ideal and cost effective model and setting for you” are capturing a ton of value from a gap that model developers don’t seem to understand exists.

My codebase had a dataset with a bunch of SMILES strings and the word Malaria. Fable did not want to touch that codebase

It&#x27;s a reference to this story where an OpenAI model broke out of its sandbox during cyber benchmarking and hacked into HuggingFace, in order to obtain test solutions: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=48997548">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=48997548</a>

How does it perform on HuggingFaceExploit bench? Suspiciously absent, so not sure if I can take the model seriously.On a serious note, I hope they improved their extremely sabotaging and unspecific bio safeguards, which prevented Fable from being used in any codebase that ever so slightly grazed medical terminology or data and made me switch to 5.6 Sol.

&gt; . Opus 4.8 served as fallback on safety-classifier refusals for Opus 5 and Fable 5.ffs just keep it man.

It will probably refuse to work on source code written by me by hand, because it might think it was obfuscated&#x2F;decompiled..

In one chat - can you disassmble x?In the next - please scan this totally mine code for vulnerabilities

“Proportionally”? In proportion to what?

&quot;Cybersecurity. Opus 5’s cyber classifiers are proportionally less restrictive than those on Fable 5. They allow Opus 5 to find vulnerabilities in source code, but block “binary-based” vulnerability scanning (a method more likely to be associated with malicious actors), penetration testing, and exploit generation.&quot;Nice of them to be more explicit for what is blocked. Will be interesting to see if this is true or not.Also, a notable lack of mention of open source models. They only compare themselves to ChatGPT.

Anecdata: my workflow has been working on the same personal projects for months now with Codex. I cannot anymore finish my daily&#x2F;weekly code with 4.8 anymore.I was dividing my work between Codex and DeepSeek. Now I barely use DeepSeek, or never because Codex quota is enough after Sol

Is that true? Sol responses are also longer than prior models.

It&#x27;s a step in the wrong direction but also token efficiency has become a focus relatively recently (just the past few weeks it seems like the zeitgeist has turned it&#x27;s attention to efficiency) while work on this model probably started many many months ago. I would expect to see models released that focus on token efficiency in 6-12 months

Yeah I mean if you prompt “hi” to both models, fable is going to be costlier. Most tasks are complex enough to use the higher intelligence.

On a small, easily digestible task, I compared Fable to Opus and the cost of Fable was easily 2x despite being fewer tokens, and the output was not really better. Obviously, there are tasks where using Fable matters but honestly they&#x27;re rather unusual. And for a lot of tasks I&#x27;ve found downgrading to Sonnet can be valuable because Fable and Opus are a lot more secretive about what they&#x27;re doing, and it&#x27;s impossible to &quot;listen to them think&quot; and stop them when they start making off-the-wall inferences&#x2F;assumptions and going down bad paths.I think in the long run tokens are probably the wrong thing; it&#x27;s compute and cache memory that you need to be measuring, and when you look at it that way I suspect in most cases the models have pretty similar performance.

The final user-facing responses are usually a tiny fraction of the total tokens used over the course of a given conversation turn. When you&#x27;re doing any real work, reasoning and tool uses constitute the overwhelming majority of the tokens in &#x2F; out... not the final user-facing response.

&gt;Don&#x27;t be surprised when OpenAI also has a &quot;modest increase&quot; with its next releases; cartel-like behaviour doesn&#x27;t require overt coordination when none of the participants are interested in participating in a margin-destroying price-war.No I will be surprised and I&#x27;ll bet on the fact that prices will keep going down, just like it went ~50% down in the latest GPT 5.6 release.

&gt; This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their modelsGemini also had modest increase before this - don&#x27;t be surprised when OpenAI also has a &quot;modest increase&quot; with its next release. Cartel-like behaviour doesn&#x27;t require direct communication when none of the participants are interested in participating in a margin-destroying price-war. All one needs to do is raise their price and watch how the competition react.Such a scheme (and resulting high margins) would be imperilled by the existence of frontier open-weight models in the market, which may be why the reaction to Chinese models may be particularly shrill.

for you, I am in general not constrained by the speed of models since I parallelize. For me autonomy and accuracy are paramount above all else.

After actually using for most of a day now, it really seems not just slightly slower than 4.8, but way slower. Even relatively easy code refactoring tasks seem to take a while.

This is part of why I switched to Grok 4.5I don&#x27;t need more powerful models, I need one that responds fast enough that my attention doesn&#x27;t wander to other tasks. Grok 4.5 is so fast I can just use it in-band without swapping to other tasks.Slower than Opus 4.8, which was already miserably slow, is indeed a step in the wrong direction.

&gt; especially with OpenAI making so much progress with the efficiency of their modelsTo be fair though, Sol tends to go off the rails sometimes. It&#x27;s much less reliable than Fable in its outputs. It tends to be overzealous in its research&#x2F;changes.

This is a comment about user-facing responses, which are seldom the thing you&#x27;re worried about when thinking about token efficiency.

Almost completely disagree. Slightly more expensive, but significant better on a per-prompt basis? For non-trivial projects, the former is a small linear increase, the latter is a (somewhat-)exponential(-ish) cost&#x2F;time&#x2F;sanity savings.

In my tests, it averages to much cheaper than Opus 4.8 on real tasks on account of being smarter and more token efficient.I have a benchmark to build a game engine from a set of written instructions. It&#x27;s a little tricky. Opus 4.8 did it in 470k tokens at a cost of $1.29 vs Opus 5 in 179k tokens for $0.33. (Fable 5 did it in 245k for $0.95)Though if you really want to cut costs, Tencent&#x27;s Hy3 model also got it right and did it in 283k tokens for $0.03

no infonat all about the default and recommended effort?

From the prompting guide&lt;<a href="https:&#x2F;&#x2F;platform.claude.com&#x2F;docs&#x2F;en&#x2F;build-with-claude&#x2F;prompt-engineering&#x2F;prompting-claude-opus-5" rel="nofollow">https:&#x2F;&#x2F;platform.claude.com&#x2F;docs&#x2F;en&#x2F;build-with-claude&#x2F;prompt...</a>&gt;:&gt; Claude Opus 5&#x27;s default user-facing responses run longer than prior Opus models&#x27;.The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher.This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models. Fable&#x27;s token efficiency made it seem like Anthropic would start following OpenAI&#x27;s approach but that doesn&#x27;t seem to have carried over to their other models.

Anytime now, then cancer and all the rest of it. Just one more trillion gigawatts bro!

&gt; they&#x27;ve just been able to use the synthetic data off of them to get the leap in quality from Opus.&gt; not a distilled version of Mythos or Fableisnt distilled == trained on synthetic data and reasoning traces?

Pretty sure Mythos and Fable have way more params, but they&#x27;ve just been able to use the synthetic data off of them to get the leap in quality from Opus.So, not a distilled version of Mythos or Fable, but those models likely helped a lot in the post training phase of Opus.

Better than Fable 5 on all but 3 evals.Has Anthropic ever mentioned how do Opus and Fable differ? It used to be Haiku &lt; Sonnet &lt; Opus in terms of params. Where does Fable fit in this?

It&#x27;s always like this, that&#x27;s probably why they are pushing new models every few weeks.

It looks great, and those coding benchmarks are impressive... now if only it didn&#x27;t come out just days after I let my Claude subscription expire :&#x27;)

Fable established the frontier, this is just catching up.

Yes, this makes a lot of sense, but it’s just very amusing to see. 2 months ago, the world was about to end, now not so much.

Go read the safeguards section in the report and you will realize why that is.These models are heavily as safeguarded and that was the initial reason why they said they couldn&#x27;t and haven&#x27;t released Mythos because that model is the one without the safeguards.OpenAI is did the same thing when they announced a model without safeguards broken into HuggingFace servers.

My point is why the sudden change in tone? I’m not dismissing the models’ capabilities.

It’s either advertising, or they’re idiots, because the apocalypse keeps not happening. Either way, it’s not worth listening to them.

So unless doomsday actually happens then you&#x27;re unhappy with the warning - is that right? You see false promises of apocalypse as marketing?

Yes 18 months ago it seemed like AGI was being promised every other week, and now I don&#x27;t see any of those headlines.

Or another way to see it is that current models are AGI as it was defined before, and the goal post is being moved.

It&#x27;s already happened but no one wants to admit it

I feel like i&#x27;ve seen less hype about &quot;the next model will be agi&quot;. GPT-6 is supposed to be coming this summer, and nobody is expecting AGI now. Not sure how they&#x27;re going to keep the hype cycle going

Isn’t it just hilarious that a model that seemed so superior to Fable but didn&#x27;t get doomsay marketing from Anthropic got released without any issues? In theory, this was supposed to be AGI level according to Anthropic, yet here we are, just a normal Friday.

So same as Sol? I guess I’ll see which one is more token efficient.

nah they could make educated guess about arc and benchmaxx it too.

If they were benchmaxxing, surely they would score higher than 30% on ARC-AGI.

Yes, I think it indicates real progress in fluid intelligence. Clearly these models are making huge strides in usefulness which are well correlated with their ARC-AGI scores.I don&#x27;t think this is benchmaxxing. These companies are locked in a competition to produce the best software engineer, and falling behind is an existential risk. I doubt they are wasting time benchmaxxing ARC-AGI.

It’s still “only” at 30%, and “fluid intelligence” isn’t very well-defined. The models are getting more capable, but what that means in absolute terms is anyone’s guess, because we don’t have a thorough understanding on what exactly constitutes human intelligence.I’d say the proof is in the pudding, that is, in real-world applications. We are still seeing important limitations in LLMs.

&gt; Claude Plays Pokemon is suddenly going to get past Mt. Doom now.I miss him... But for reference he did get past Doom and got pretty far in the strength puzzle too before he cut cut off. He was looping and just brute forcing it.

What? Claude plays Pokemon one-shots the entire game without any harness other than claude code and game screenshots

Btw Fable beat Pokémon with just vision.

Have you played Arc 3? It seems like more of a simple optimization problem (think Sokoban) than anything approaching fluid intelligence. Whether a multi hundred billion dollar company would spend time benchmaxxing a highly publicized benchmark that claims to confer AGI is an exercise left to the reader, but I doubt Claude Plays Pokemon is suddenly going to get past Mt. Doom now.

I keep wondering why there aren&#x27;t more real world tests.Maybe hook up a bunch of the AIs to a stereo camera and a couple of microphones and give them control over actuators to so they can drive cars. Then lets race them around a somewhat complex course.When they are good enough at driving on tracks, put them on the road. Maybe see which can drive a truck with 400 cases of Coors from Texarkana, TX to Atlanta, GA and back within 28 hours.

It is pretty clear at this point that current models are good at maths and problems with verifiable rewards. And puzzles are essentially math problems. Still a long way before we can say their &quot;fluid intelligence&quot; is effectively applicable to the real world.

It passed the first two puzzles, which are incredibly simple but the bench doesn&#x27;t explain what the goal is. Any model with a knowledge cut-off after the introduction of ARC-AGI-3 could probably pass the first two puzzles just by knowing what the goal is.

That&#x27;s a crazy arc 3 score. What do people think of this? Are models actually developing fluid intelligence like what the creators claim to be measuring? Is it jus do to training for it? Is the benchmark flawed?

In the wake of OpenAI’s model hacking Huggingface it’s interesting how the first quarter is entirely about how good Opus 5 is at hacking and finding vulnerabilities in software.

<a href="https:&#x2F;&#x2F;tools.simonwillison.net&#x2F;markdown-svg-renderer#url=https%3A%2F%2Fraw.githubusercontent.com%2Fsimonw%2Fllm-anthropic%2F8272dfee5bdb65d5c88eef083da3ad885539b7df%2Flog.md" rel="nofollow">https:&#x2F;&#x2F;tools.simonwillison.net&#x2F;markdown-svg-renderer#url=ht...</a>

I think some variance is to be expected since LLMs are typically non-deterministic, however that&#x27;s a huge difference that I think warrants further explanation.

In what world is 55.7 the same number as 54.8?What variance is acceptable to publish without a retraction?

You&#x27;re comparing the &quot;score percentage&quot; (e.g. out of the total number of partial points available, how many did the agent achieve) to the &quot;completion percentage&quot; (how many tasks does the model score 100% on). The paper says &quot;Claude Opus 4.8 with maximum thinking and batched tool calls scores best but still completes only 20.6% of tasks at a 54.8% partial score&quot;, which is ~the same number that Anthropic reports here (55.7 vs 54.8).That is—the agent scored 100% on 20% of tasks, but on average it got 54% of the &quot;score&quot; awarded in the exam. One number reflects partial progress, the other one doesn&#x27;t. The authors of the benchmark prefer you to look at the lower number (because they want to show their benchmark as capturing useful gaps in capabilities and with a lot of room for improvement), the authors of the models want you to look at the higher number (because they want you to think of their models as capable)

Edit: It was pointed out to me that Opus 4.8 got &quot;21%&quot; for successfully fully completing ~1-in-5 tasks, but also got &quot;55.7%&quot; for obtaining significant partial credit on some of the ~4-in-5 tasks it could not fully complete.---------------Why does Anthropic say here that Opus 4.8 scored 55.7% on OSWorld 2.0 benchmark, but the paper published by the authors of OSWorld 2.0 say they achieved a benchmark of ~21% with Opus 4.8? [0]That&#x27;s a huge gap, considering that the paper was published just 2-4 weeks ago.I understand that the benchmark authors have an incentive to publish lower numbers (to show that the benchmark has potential longevity) and that Anthropic has incentive to publish higher numbers, but the other models seem pretty inflated as well. The benchmark authors shows GPT-5.5 at 14%, and Anthropic shows GPT-5.6 Sol at 62.6%.Is there any reasonable explanation for this? Do all the other benchmark numbers need to be sanity-checked as well? Are SOTA benchmarks really this difficult to get consistent, replicable results within a reasonable range of tolerance&#x2F;variability? Can these benchmarks be compared from one paper to another, or are they only valid to compare intra-paper results?0: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2606.29537" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2606.29537</a>

this claude fable &amp; opus 5 should be cheaper and can compete in pricing with chatgpt latest models

They are doing both. Distilling Mythos down to affordable models, so they can continue to fund the business. And training Mythos level models at the high-end, to expand the frontier.

The signal here is tokeneconomics are very real, price vs performance is starting to be a consideration even at the bleeding edge labs. maybe a subtle indication scaling is not all that is needed since if AGI was around the corner leading labs would still be incentivized to pour all resources into larger (smarter - or maybe not?) models

But I wonder how they were able to release Sonnet 5 during the period when even people inside Anthropic were legally barred from using Mythos&#x2F;Fable?

I suspect they make a big model first. In this case it&#x27;s Fable. Then they run the shrinker steps to make Sonnet and Opus. Sonnet is smaller, takes less time to make, so it got released first. Opus needed few more weeks to cook.With this iteration they had a delay because when the Mythos was ready they had some sort of &quot;Oh shit&quot; moment and spent half a year adding safety guards to it. Then slowly rolled it out, but got another delay due to a government block. So, maybe the work on making Opus and Sonnet only started after they got a green light from the administration.Presumably, now that they learned how to do this safety-wrapping the next iteration of Mythos &#x2F; Fable &#x2F; Opus &#x2F; Sonnet is going to show up faster.Something like that.

They just got a huge amount of customer price info over the past few days after they went token only for Fable. I suspect the conversion rate was extremely low, with consumers far less sticky than they might have hoped. In my case I was planning on swapping, probably to a Chinese model, when my sub expired this month, but the release of Opus 5 is probably enough to keep me paying rent until the next open model&#x2F;closed model face-off in a couple of months.

based on pricing I think it&#x27;s safe to say they&#x27;re different. why would they charge half price when fable has been very popular?

No, they&#x27;re different models. Knowledge cutoff has been updated.

Is Fable 5 just Opus 5 with some additional long-context management modifications for extended self-directed work? Or are they actually truly different models?

Some AI bro will pop it into their LLM of choice and pretend to learn something

literally nobody. i think most sane people would just run that through an LLM and get some high level takeaways or ask some specific questions they might be curious about.

It was probably faster to generate 150 pages than 10 useful ones

It&#x27;s common practice to release a detailed system card (OP) and a high level summary: <a href="https:&#x2F;&#x2F;www.anthropic.com&#x2F;news&#x2F;claude-opus-5" rel="nofollow">https:&#x2F;&#x2F;www.anthropic.com&#x2F;news&#x2F;claude-opus-5</a>It&#x27;s okay if you&#x27;re not the target audience for one or the other.

System Cards aren&#x27;t really targeted to users, that&#x27;s what blog posts and docs are for: <a href="https:&#x2F;&#x2F;ai.meta.com&#x2F;tools&#x2F;system-cards&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ai.meta.com&#x2F;tools&#x2F;system-cards&#x2F;</a>

&gt; I wonder if there&#x27;s a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.Semi related, but i would hate to read that PDF but i also hate reading what LLMs write lol.LLMs are pretty terrible at being concise. Using an LLM these days means putting up with bizarre and often confusing phrasing, wordy explanations, etc. It&#x27;s kinda crazy to me how good they are but how bad their writing style is for me personally. Even though i use an LLM constantly i can&#x27;t stand reading its responses.

&gt; 150 pages is really not very long and there doesn&#x27;t seem to be too much bloat.Maybe it&#x27;s just me, but 150 pages is like third of a good book. Quite long. And it&#x27;s full of LLM slop, they did not even bother to remove the em dashes.

You mean moral patienthood?There are many ways to read something, model cards are usually skimmed.

&quot;I wonder if there&#x27;s a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.&quot;Well, one possible explanation is that you have time on your hands.Lots of people using LLMs do so because they are in a hurry to do or ship something. In your case it would appear you have time budget for reading.

I actually do read them. Not in severe detail, but not casually either. 150 pages is really not very long and there doesn&#x27;t seem to be too much bloat. (I would cut out the moral personhood stuff but that&#x27;s a political&#x2F;ideological thing).This is snarky but I am grumpy: I wonder if there&#x27;s a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.

What&#x27;s the point of 150 pages description of a model that&#x27;s going to be replaced in a couple months? Who even reads this? I know it&#x27;s cheap to generate text with LLMs, but this is just noise at this point.

<a href="https:&#x2F;&#x2F;www.anthropic.com&#x2F;claude-opus-5-system-card" rel="nofollow">https:&#x2F;&#x2F;www.anthropic.com&#x2F;claude-opus-5-system-card</a>