Memory could be amazing for coding in large codebases. Web search could be great for finding docs on dependencies as well. Are these features integrated with Claude Code though?

livestream here: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;EvtPBaaykdo" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;EvtPBaaykdo</a>my highlights:1. Coding ability: &quot;Claude Opus 4 is our most powerful model yet and the best coding model in the world, leading on SWE-bench (72.5%) and Terminal-bench (43.2%). It delivers sustained performance on long-running tasks that require focused effort and thousands of steps, with the ability to work continuously for several hours—dramatically outperforming all Sonnet models and significantly expanding what AI agents can accomplish.&quot; however this is Best of N, with no transparency on size of N and how they decide the best, saying &quot;We then use an internal scoring model to select the best candidate from the remaining attempts.&quot; Claude Code is now generally available (we covered in <a href="http:&#x2F;&#x2F;latent.space&#x2F;p&#x2F;claude-code" rel="nofollow">http:&#x2F;&#x2F;latent.space&#x2F;p&#x2F;claude-code</a> )2. Memory highlight: &quot;Claude Opus 4 also dramatically outperforms all previous models on memory capabilities. When developers build applications that provide Claude local file access, Opus 4 becomes skilled at creating and maintaining &#x27;memory files&#x27; to store key information. This unlocks better long-term task awareness, coherence, and performance on agent tasks—like Opus 4 creating a &#x27;Navigation Guide&#x27; while playing Pokémon.&quot; Memory Cookbook: <a href="https:&#x2F;&#x2F;github.com&#x2F;anthropics&#x2F;anthropic-cookbook&#x2F;blob&#x2F;main&#x2F;tool_use&#x2F;memory_cookbook.ipynb">https:&#x2F;&#x2F;github.com&#x2F;anthropics&#x2F;anthropic-cookbook&#x2F;blob&#x2F;main&#x2F;t...</a>3. Raw CoT available: &quot;we&#x27;ve introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access.&quot;4. haha: &quot;We no longer include the third ‘planning tool’ used by Claude 3.7 Sonnet. &quot; &lt;- psyop?5. context caching now has a premium 1hr TTL option: &quot;Developers can now choose between our standard 5-minute time to live (TTL) for prompt caching or opt for an extended 1-hour TTL at an additional cost&quot;6. <a href="https:&#x2F;&#x2F;www.anthropic.com&#x2F;news&#x2F;agent-capabilities-api" rel="nofollow">https:&#x2F;&#x2F;www.anthropic.com&#x2F;news&#x2F;agent-capabilities-api</a> new code execution tool (sandbox) and file tool

&gt; Really wish I could say more.Have you used it?I liked Claude 3.7 but without context this comes off as what the kids would call &quot;glazing&quot;

Game changer is table stakes now, tell us something new.

Run claude cli from inside of VS Code terminal.<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44064082">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44064082</a>

Thanks, I found it in node_modules&#x2F;@anthropic-ai&#x2F;claude-code&#x2F;vendor

When you install claude code (or update it) there is a .vsix file in the same area where claude bin is.

Doesn&#x27;t work in Cursor over ssh. I found the VSIX in node_modules&#x2F;@anthropic-ai&#x2F;claude-code&#x2F;vendor and installed it manually.

Run claude in the terminal in VSCode (or cursor) and it will install automatically!

Show diffs in my editor windows, like Cursor agent mode does, is what I&#x27;m hoping.

Claude Code as a tool call, from Copilot‘s own agent (agent in an agent) seems to be working well. Peter Steinberger made an MCP that does this: <a href="https:&#x2F;&#x2F;github.com&#x2F;steipete&#x2F;claude-code-mcp">https:&#x2F;&#x2F;github.com&#x2F;steipete&#x2F;claude-code-mcp</a>

In the video: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;live&#x2F;EvtPBaaykdo?si=m2GWFMSItZeb8I9r&amp;t=3333" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;live&#x2F;EvtPBaaykdo?si=m2GWFMSItZeb8I9r...</a>

What would this do other than run Claude Code in the same directory you have open in VSC?

Have you upgraded your binary recently. I feel like this was true 1-2 months ago, but not anymore.

Claude code is a poorer version of aider or cline in VScode. I have better results using them than using claude code alone.

Let us know if you were able to get it installed. You need to run claude inside the vscode (or cursor&#x2F;windsurf) terminal for it to auto-install.

Ooh, VS Code integration for Claude Code sounds nice. I do feel like Claude Code works better than the native Cursor agent mode.Edit: How do you install it? Running `&#x2F;ide` says &quot;Make sure your IDE has the Claude Code extension&quot;, where do you get that?

Good, I was starting to get uncomfortable with how hard Gemini has been dominating latelyETA: I guess Anthropic still thinks they can command a premium, I hope they&#x27;re right (because I would love to pay more for smarter models).&gt; Pricing remains consistent with previous Opus and Sonnet models: Opus 4 at $15&#x2F;$75 per million tokens (input&#x2F;output) and Sonnet 4 at $3&#x2F;$15.

heh, I just wrote a small hit piece about all the disappointments of the models over the last year and now the next day there is a new model. I&#x27;m going to assume it will still get you only to 80% ( ͡° ͜ʖ ͡°)

It’s up on their pricing page: <a href="https:&#x2F;&#x2F;www.anthropic.com&#x2F;pricing" rel="nofollow">https:&#x2F;&#x2F;www.anthropic.com&#x2F;pricing</a>

Thanks. I looked a couple minutes ago and couldn&#x27;t see it. For anyone curious, pricing remains the same as previous Anthropic models.

Yeah it&#x27;s live on the pricing page:<a href="https:&#x2F;&#x2F;www.anthropic.com&#x2F;pricing#api" rel="nofollow">https:&#x2F;&#x2F;www.anthropic.com&#x2F;pricing#api</a>Opus 4 is $15 &#x2F; m tokens in, $75 &#x2F; MTok out
Sonnet 4 is the same $3 &#x2F; MTok in, $15 &#x2F; MTok out

From the linked post:&gt; Pricing remains consistent with previous Opus and Sonnet models: Opus 4 at $15&#x2F;$75 per million tokens (input&#x2F;output) and Sonnet 4 at $3&#x2F;$15.

Claude names their models based on size&#x2F;complexity:- Small: Haiku- Medium: Sonnet- Large: Opus

&gt; Try Claude Sonnet 4 today with Claude Opus 4 on paid plans.Wait, Sonnet 4? Opus 4? What?

Aider is not just leetcode exercises I think? livecodebench is leetcode exercises though.

Aider uses a dataset of 500 GitHub issues, so not LeetCode-style work.

Are these just leetcode exercises? What I would like to see is an independent benchmark based on real tasks in codebases of varying size.

I&#x27;ll look at it when this shows up on <a href="https:&#x2F;&#x2F;aider.chat&#x2F;docs&#x2F;leaderboards&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aider.chat&#x2F;docs&#x2F;leaderboards&#x2F;</a> I feel like keeping up with all the models is a full time job so I just use this instead and hopefully get 90% of the benefit I would by manually testing out every model.

The option just shown up in Copilot settings page for me

The keynote confirms it is available now.

Anyone see news of when it’s planned to go live in copilot?

And IMO it has a long way to go. There is a lot of nuance when orchestrating dependencies that can cause subtle errors in an application that are not easily remedied.For example a lot of llms (I&#x27;ve seen it in Gemini 2.5, and Claude 3.7) will code non-existent methods in dynamic languages. While these runtime errors are often auto-fixable, sometimes they aren&#x27;t, and breaking out of an agentic workflow to deep dive the problem is quite frustrating - if mostly because agentic coding entices us into being so lazy.

Personally, I don&#x27;t believe AI is ever going to get to that level. I&#x27;d love to be proven wrong, but I really don&#x27;t believe that an LLM is the right tool for a job that requires novel thinking about out of the ordinary problems like all the weird edge cases and poor documentation that comes up when trying to upgrade old software.

I think this type of thing needs agent which has access to the documentation to read about nuances of the language and package versions, definitely a way to investigate types, interfaces. Problem is that training data has so much mixed data it can easily confuse the AI to mix up versions, APIs etc.

That&#x27;s kind of my benchmark for whether or not these models are useful. I&#x27;ve got a project that needs some extensive refactoring to get working again. Mostly upgrading packages, but also it will require updating the code to some new language semantics that didn&#x27;t exist when it was written. So far, current AI models can make essentially zero progress on this task. I&#x27;ll keep trying until they can!

And they might all be right!&gt; This tech could lead to...I don&#x27;t think he&#x27;s saying this is the version that will suddenly trigger a Renaissance. Rather, it&#x27;s one solid step that makes the path ever more promising.Sure, everyone gets a bit overexcited each release until they find the bounds. But the bounds are expanding, and the need for careful prompt engineering is diminishing. Ever since 3.7, Claude has been a regular part of my process for the mundane. And so far 4.0 seems to take less fighting for me.A good question would be when can AI take a basic prompt, gather its own requirements and build meaningful PRs off basic prompt. I suspect it&#x27;s still at least a couple of paradigm shifts away. But those seem to be coming every year or faster.

It could be! But that&#x27;s also what people said about all the models before it!

Especially since the EU just made open source contributors liable for cybersecurity (Cyber Resilience Act). Just let AI contribute and ur good

&gt; I am incredibly eager to see what affordable coding agents can do for open source :)Oh, we know exactly what they will do: they will drive devs insane: <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ExperiencedDevs&#x2F;comments&#x2F;1krttqo&#x2F;my_new_hobby_watching_ai_slowly_drive_microsoft&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ExperiencedDevs&#x2F;comments&#x2F;1krttqo&#x2F;my...</a>

I am incredibly eager to see what affordable coding agents can do for open source :) in fact, I should really be giving away CheepCode[0] credits to open source projects. Pending any sort of formal structure, if you see this comment and want free coding agent runs, email me and I’ll set you up![0] My headless coding agents product, similar to “assign to copilot” but works from your task board (Linear, Jira, etc) on multiple tasks in parallel. So far simple&#x2F;routine features are already quite successful. In general the better the tests, the better the resulting code (and yes, it can and does write its own tests).

Until it pushes a severe vulnerability which takes a big service doen

&gt; having package upgrades and other mostly-mechanical stuff handled automaticallyThose are already non-issues mostly solved by bots.In any case, where I think AI could help here would be by summarizing changes, conflicts, impact on codebase and possibly also conduct security scans.

I don&#x27;t see how a LLM could do better than a bot, eg renovate

“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.”Maybe this model will push the “Assign to CoPilot” closer to the dream of having package upgrades and other mostly-mechanical stuff handled automatically. This tech could lead to a huge revival of older projects as the maintenance burden falls.

One strategy that also works is to have 2 separate &quot;sessions&quot;, have one write code and one write tests. Forbid one to change the other&#x27;s &quot;domain&quot;. Much better IME.

I once saw probably 10 iterations to fix a broken test, then it decided that we don&#x27;t need this test at all, and it tried to just remove it.IMO, you either write tests and let it write implementation or write implementation and let it write tests. Maybe use something to write tests, then forbid &quot;implementor&quot; to modify them.

In my experience, when presented with a failing test it would simply try to make the test pass instead of determining why the test was failing. Usually by hard coding the test parameters (or whatever) in the failing function... which was super annoying.

&gt; we’ve significantly reduced behavior where the models use shortcuts or loopholes to complete tasks. Both models are 65% less likely to engage in this behavior than Sonnet 3.7 on agentic tasksSounds like it’ll be better at writing meaningful tests

Gemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don&#x27;t mean much for comparing Claude to Gemini.

Okay, wait though like I want to know the full transcript because that actually is a better &#x2F; softer benchmark if you measure in terms of the necessary human input.

That Google IO slide was somewhat misleading as the maintainer of Gemini Plays Pokemon had a much better agentic harness that was constantly iterated upon throughout the runtime (e.g. the maintainer had to give specific instructions on how to use Strength to get past Victory Road), unlike Claude Plays Pokemon.The Elite Four&#x2F;Champion was a non-issue in comparison especially when you have a lv. 81 Blastoise.

It&#x27;s a fun benchmark, like simonw&#x27;s pelican riding a bike. Sometimes fun is the best metric.

from a foundational research perspective, the pokemon benchmark is one of the most important ones.these models are trained on a static task, text generation, which is to say the state they are operating in does not change as they operate. but now that they are out we are implicitly demanding they do dynamic tasks like coding, navigation, operating in a market, or playing games. this are tasks where your state changes as you operatean example would be that as these models predict the next word, the ground truth of any further words doesnt change. if it misinterprets the word bank in the sentence &quot;i went to the bank&quot; as a river bank rather than a financial bank, the later ground truth wont change, if it was talking about the visit to the financial bank before, it will still be talking about that regardless of the model&#x27;s misinterpretation. But if a model takes a wrong turn on the road, or makes a weird buy in the stock market, the environment will react and change and suddenly, what it should have done as the n+1th move before isnt the right move anymore, it needs to figure out a route of the freeway first, or deal with the FOMO bullrush it caused by mistakenly buying alot of stockwe need to push against these limits to set the stage for the next evolution of AI, RL based models that are trained in dynamic reactive environments in the first place

&gt; Where is the killer app?My man, ChatGPT is the sixth most visited website in the world right now.

This is a weirdly cherry-picked example. The gaming approach was also the premise of DeepMind&#x27;s AI efforts in 2016, which was nine years ago. Regardless of what you think about the utility of text (code), video, audio, and image generation, surely you think that their progress on the protein-folding problem and weather prediction have been useful to society?What counts as a killer app to you? Can you name one?

Right, but on the other hand... how is it even useful? Let&#x27;s say it can beat the game, so what? So it can (kind of) summarise or write my emails - which is something I neither want nor need, they produce mountains of sloppy code, which I would have to end up fixing, and finally they can play a game? Where is the killer app? The gaming approach was exactly the premise of the original AI efforts in the 1960s, that teaching computers to play chess and other &#x27;brainy&#x27; games will somehow lead to development of real AI. It ended as we know in the AI nuclear winter.

Not sure &quot;original concept&quot; is quite right, given it had been tried earlier, e.g. here&#x27;s a 2023 attempt to get gpt-4-vision to play pokemon, (it didn&#x27;t really work, but it&#x27;s clearly &quot;the concept&quot;)<a href="https:&#x2F;&#x2F;x.com&#x2F;sidradcliffe&#x2F;status&#x2F;1722355983643525427" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;sidradcliffe&#x2F;status&#x2F;1722355983643525427</a>

I disagree - this is all an homage to Twitch Plays Pokemon, which was a noteworthy moment in internet culture&#x2F;history.<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Twitch_Plays_Pok%C3%A9mon" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Twitch_Plays_Pok%C3%A9mon</a>

Claude Plays Pokemon was the original concept and inspiration behind &quot;Gemini Plays Pokemon&quot;. Gemini arguably only did better because it had access to a much better agent harness and was being actively developed during the run.See: <a href="https:&#x2F;&#x2F;www.lesswrong.com&#x2F;posts&#x2F;7mqp8uRnnPdbBzJZE&#x2F;is-gemini-now-better-than-claude-at-pokemon" rel="nofollow">https:&#x2F;&#x2F;www.lesswrong.com&#x2F;posts&#x2F;7mqp8uRnnPdbBzJZE&#x2F;is-gemini-...</a>

Sooo... it can play Pokemon. Feels like they had to throw that in after Google IO yesterday. But the real question is now can it beat the game including the Elite Four and the Champion. That was pretty impressive for the new Gemini model.

Well Claude 3.7 definitely did the one where it was supposed to process a file and it finally settled on `fs.copyFile(src, dst)` which I think is pro-level interaction. I want those $0.95 back.But I love you Claude. It was me, not you.

Well, at least it doesn&#x27;t distract your coworkers, disrupting their flow

Which sort of workday? The sort where you rewrite your code 8 times and end the day with no marginal business value produced?

Easy, I can also make a nanoGPT run for 7 hours when inferring on a 68k, and make it produce as much value as I usually do.

I can write an algorithm to run in a loop forever, but that doesn&#x27;t make it equivalent to infinite engineers. It&#x27;s the output that matters.

How much does it cost to have it run for 7 hours straight?

&gt;Rakuten validated its capabilities with a demanding open-source refactor running independently for 7 hours with sustained performance.From their customer testimonials in the announcement, more below&gt;Cursor calls it state-of-the-art for coding and a leap forward in complex codebase understanding. Replit reports improved precision and dramatic advancements for complex changes across multiple files. Block calls it the first model to boost code quality during editing and debugging in its agent, codename goose, while maintaining full performance and reliability. Rakuten validated its capabilities with a demanding open-source refactor running independently for 7 hours with sustained performance. Cognition notes Opus 4 excels at solving complex challenges that other models can&#x27;t, successfully handling critical actions that previous models have missed.&gt;GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot. Manus highlights its improvements in following complex instructions, clear reasoning, and aesthetic outputs. iGent reports Sonnet 4 excels at autonomous multi-feature app development, as well as substantially improved problem-solving and codebase navigation—reducing navigation errors from 20% to near zero. Sourcegraph says the model shows promise as a substantial leap in software development—staying on track longer, understanding problems more deeply, and providing more elegant code quality. Augment Code reports higher success rates, more surgical code edits, and more careful work through complex tasks, making it the top choice for their primary model.

Makes no sense to measure it in hours. You can have a slow CPU making the model run for longer.

Allegedly Claude 4 Opus can run autonomously for 7 hours (basically automating an entire SWE workday).

&quot;We&#x27;re having trouble connecting to the model provider&quot;A bit busy at the moment then.

Looks like both opus and sonnet are already in Cursor.

Nobody cares about lmarena anymore? I guess it&#x27;s too easy to cheat there after a llama4 release news?

You can likely set up an MCP server that handles this.

I have the Claude Windows app, how long until it can &quot;see&quot; what&#x27;s on my screen and help me code&#x2F;debug?

pretty sure the cringe doesn&#x27;t stop there. It wouldn&#x27;t surprise me if this is not the only thing that they are attempting to game and hide from their users.The Max subscription with fake limits increase comes to mind.

&gt; Finally, we&#x27;ve introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access.Extremely cringe behaviour. Raw CoTs are super useful for debugging errors in data extraction pipelines.After Deepseek R1 I had hope that other companies would be more open about these things.

I guess because they haven&#x27;t been releasing all three models of the same version in a while now. We&#x27;ve only had Sonnet updates, so the version first didn&#x27;t make sense if we had 3.5 and 3.7 Sonnet but none of the others.

Wonder why they renamed it from Claude &lt;number&gt; &lt;type&gt; (e.g. Claude 3.7 Sonnet) to Claude &lt;type&gt; &lt;number&gt; (Claude Opus 4).

My guess is that they did RLVR post-training for SWE tasks, and a smaller model can undergo more RL steps for the same amount of computation.

Do you have the link to that benchmark? Can’t see where Sonnet is highlighted.

Interesting how Sonnet has a higher SWE-bench Verified score than Opus. Maybe says something about scaling laws.

OpenIA&#x27;s Codex-1 isn&#x27;t so cool anymore. If it was ever cool.And Claude Code used Opus 4 now!

In my opinion most Anthropic models are the opposite, scoring well on benchmarks but not always way on top, but quietly excellent when you actually try to use them for stuff.

Can&#x27;t wait to hear how it breaks all the benchmarks but have any differences be entirely imperceivable in practice.

I would say yes. The jump in capability and reduction in hallucinations (at least code) to Claude 3.7 from ChatGPT (even o3) is immediately noticeable in my experience. Same goes for gemini which was even better in some ways until perhaps today.

Kinda interesting, as I&#x27;ve found 4o better than o4-mini-high for most of my coding.  And while it&#x27;s mind blowing that they can do what they can do, the code itself coming out the other end has been pretty bad, but good enough for smaller snippets and extremely isolated features.

My mind has been blown using ChatGPT&#x27;s o4-mini-high for coding and research (it knowledge of computer vision and tools like OpenCV are fantastic). Is it worth trying out all the shiny new AI coding agents ... I need to get work done?

I don&#x27;t think there&#x27;s any real evidence that any of the major companies are going out of their way to cheat the benchmarks. Problem is that, unless you put a lot of effort into avoiding contamination, you will inevitably end up with details about the benchmark in the training set.

Hasn&#x27;t it been proven many times that all those companies cheat on benchmarks?I personally couldn&#x27;t care less about them, especially when we&#x27;ve seen many times that the public&#x27;s perception is absolutely not tied to the benchmarks (Llama 4, the recent OpenAI model that flopped, etc.).

Or someone who has been a developer for a decade plus trying to use these models on actual existing code bases, solving specific problems. In my experience, they waste time and money.

People&#x27;s interpretation of benchmarks will largely depend on whether they believe they will be better or worse off by GenAI taking over SWE jobs.  Think you&#x27;d need someone outside the industry to weigh in to have a real, unbiased view.

but as soon as you test on your private testing set you&#x27;re sending it to their servers so they have access to itso effectively you can only guarantee a single use stays private

Well-designed benchmarks have a public sample set and a private testing set. Models are free to train on the public set, but they can&#x27;t game the benchmark or overfit the samples that way because they&#x27;re only assessed on performance against examples they haven&#x27;t seen.Not all benchmarks are well-designed.

kind of reminds me how they said they were increasing platform capabilities with Max and actually reduced them while charging a ton for it per month. Talk about a bait and switch. Lord help you if you tried to cancel your ill advised subscription during that product roll out as well - doubly so if you expect a support response.

The fact that people like you are still posting like this after Veo 3 is wild.  Nothing could possibly be forcing you to hold onto that opinion, yet you come out in drones in every AI thread to repost it.

Benchmark scores are marketing fluff. Just like the rest of this article with alleged praises from early adopters, and highly scripted and edited videos.AI companies are grasping at straws by selling us minor improvements to stale technology so they can pump up whatever valuation they have left.

&gt; Obviously with immense funding at stakes, companies have every incentive to game the benchmarks, and the loss of goodwill from gaming the system doesn&#x27;t appear to have much consequences.Claude 3.7 Sonnet was consistently on top of OpenRouter in actual usage despite not gaming benchmarks.

I&#x27;m curious what are others priors when reading benchmark scores. Obviously with immense funding at stakes, companies have every incentive to game the benchmarks, and the loss of goodwill from gaming the system doesn&#x27;t appear to have much consequences.Obviously trying the model for your use cases more and more lets you narrow in on actually utility, but I&#x27;m wondering how others interpret reported benchmarks these days.

DeepSeek never mopped the floor with anyone... DeepSeek was remarkable because it is claimed that they spent a lot less training it, and without Nvidia GPUs, and because they had the best open weight model for a while. The only area they mopped the floor in was open source models, which had been stagnating for a while. But qwen3 mopped the floor with DeepSeek R1.

Do people actually believe this? While I agree their open source contribution was impressive, I never got the sense they mopped the floor. Perhaps firms in China may be using some of their models but beyond learnings in the community, no dents in the market were made for the West.

Guess we have to wait till DeepSeek mops the floor with everyone again.

Yeah we really should stop focusing on model alignment. The idea that it&#x27;s more important that your AI will fucking report you to the police if it thinks you&#x27;re being naughty than that it actually works for more stuff is stupid.

Correct me if I&#x27;m wrong--my understanding is that RHLF was the difference between GPT 3 and GPT 3.5, aka the original ChatGPT.If you never used GPT 3, it was... not good. Well, that&#x27;s not fair, it was revolutionary in its own right, but it was very much a machine for predicting the most likely next word, it couldn&#x27;t talk to you the way ChatGPT can.Which is to say, I think RHLF is important for much more than just preventing PR disasters. It&#x27;s a key part of what makes the models useful.

IIRC RLHF inevitably compromises model accuracy in order to train the model not to give dangerous responses.It would make sense if the model used for train-of-though was trained differently (perhaps a different expert from an MoE?) from the one used to interact with the end user, since the end user is only ever going to see its output filtered through the public model the chain-of-thought model can be closer to the original, more pre-rlhf version without risking the reputation of the company.This way you can get the full performance of the original model whilst still maintaining the necessary filtering required to prevent actual harm (or terrible PR disasters).

it just makes it too easy to distill the reasoning into a separate model I guess. though I feel like o3 shows useful things about the reasoning while it&#x27;s happening

I&#x27;ve been thinking for a couple of months now that prompt engineering, and therefore CoT, is going to become the &quot;secret sauce&quot; companies want to hold onto.If anything that is where the day to day pragmatic engineering gets done. Like with early chemistry, we didn&#x27;t need to precisely understand chemical theory to produce mass industrial processes by making a good enough working model, some statistical parameters, and good ole practical experience. People figured out steel making and black powder with alchemy.The only debate now is whether the prompt engineering models are currently closer to alchemy or modern chemistry? I&#x27;d say we&#x27;re at advanced alchemy with some hints of rudimentary chemistry.Also, unrelated but with CERN turning lead into gold, doesn&#x27;t that mean the alchemists were correct, just fundamentally unprepared for the scale of the task? ;)

We won&#x27;t know without an official answer leaking, but a simple answer could be - people spend too much time trying to analyse those without understanding the details. There was a lot of talk on HN about the thinking steps second guessing and contradicting itself. But in practice that step is both trained by explicitly injecting the &quot;however&quot;, &quot;but&quot; and similar words and they do more processing than simply interpreting the thinking part as text we read. If the content is commonly misunderstood, why show it?

Because it&#x27;s alchemy and everyone believes they have an edge on turning lead into gold.

You are referring to the new (few days old-ish) CoT right? It’s bizzare as to why google did it, it was very helpful to see where the model was making assumptions or doing something wrong. Now half the time it feels better to just use flash with no thinking mode but ask it to manually “think”.

The Google CoT is so incredibly dumb. I thought my models had been lobotomized until I realized they must be doing some sort of processing on the thing.

The trend towards opaque is inexorable.<a href="https:&#x2F;&#x2F;noisegroove.substack.com&#x2F;p&#x2F;somersaulting-down-the-slippery-slope" rel="nofollow">https:&#x2F;&#x2F;noisegroove.substack.com&#x2F;p&#x2F;somersaulting-down-the-sl...</a>

i think it is almost certainly to prevent distillation

I have no idea what this means, can someone give the eli5?

Could the exclusion of CoT that be because of this recent Anthropic paper?<a href="https:&#x2F;&#x2F;assets.anthropic.com&#x2F;m&#x2F;71876fabef0f0ed4&#x2F;original&#x2F;reasoning_models_paper.pdf" rel="nofollow">https:&#x2F;&#x2F;assets.anthropic.com&#x2F;m&#x2F;71876fabef0f0ed4&#x2F;original&#x2F;rea...</a>&gt;We evaluate CoT faithfulness of state-of-the-art reasoning models across 6 reasoning hints presented in the prompts and find: (1) for most settings and models tested, CoTs reveal their usage of hints in at least 1% of examples where they use the hint, but the reveal rate is often below 20%, (2) outcome-based reinforcement learning initially improves faithfulness but plateaus without saturating, and (3) when reinforcement learning increases how frequently hints are used (reward hacking), the propensity to verbalize them does not increase, even without training against a CoT monitor. These results suggest that CoT monitoring is a promising way of noticing undesired behaviors during training and evaluations, but that it is not sufficient to rule them out.I.e., chain of thought may be a confabulation by the model, too. So perhaps there&#x27;s somebody at Anthropic who doesn&#x27;t want to mislead their customers. Perhaps they&#x27;ll come back once this problem is solved.

&gt; because it helped to see when it was going to go down the wrong trackIt helped me tremendously learning Zig.Seeing his chain of thought when asking it stuff about Zig and implementations let me widen the horizon a lot.

&gt; Users requiring raw chains of thought for advanced prompt engineering can contact salesSo it seems like all 3 of the LLM providers are now hiding the CoT - which is a shame, because it helped to see when it was going to go down the wrong track, and allowing to quickly refine the prompt to ensure it didn&#x27;t.In addition to openAI, Google also just recently started summarizing the CoT, replacing it with an, in my opinion, overly dumbed down summary.

But if Gemini 2.5 pro was considered to be the strongest coder lately, does SWE-bench really reflect reality?

Mathematicians don&#x27;t do high school math competitions - the benchmark in question is AIME.Mathematicians generally do novel research, which is hard to optimize for easily. Things like LiveCodeBench (leetcode-style problems), AIME, and MATH (similar to AIME) are often chosen by companies so they can flex their model&#x27;s capabilities, even if it doesn&#x27;t perform nearly as well in things real mathematicians and real software engineers do.

That&#x27;s a very contentious opinion you&#x27;re stating there. I&#x27;d say LLMs have surpassed a larger percentage of SWEs in capability than they have for mathematicians.

Nice to see that Sonnet performs worse than o3 on AIME but better on SWE-Bench. Often, it&#x27;s easy to optimize math capabilities with RL but much harder to crack software engineering. Good to see what Anthropic is focusing on.

I also use DeepSeek R1 as a daily driver. Combined with Qwen3 when I need better tool usage.Now that both Google and Claude are out, I expect to see DeepSeek R2 released very soon. It would be funny to watch an actual open source model getting close to the commercial competition.

A nice thing about Deepseek is that it is so cheap to run. It&#x27;s nice being able to explore conversational trees without getting a $12 bill at the end of it.

I&#x27;ve been initially fascinated by Claude, but then I found myself drawn to Deepseek. My use case is different though, I want someone to talk to.

It would be pretty funny if your &quot;not today. Maybe tomorrow&quot; proposition actually did happen the next day.

Did you try it? Were the results any better?

Same. And I JUST tried their GitHub Action agentic thing yesterday (wrote about it here[0]), and it honestly didn&#x27;t perform very well. I should try it again with Claude 4 and see if there are any differences. Should be an easy test[0] <a href="https:&#x2F;&#x2F;mattsayar.com&#x2F;personalized-software-really-is-coming-but-not-today-maybe-tomorrow&#x2F;" rel="nofollow">https:&#x2F;&#x2F;mattsayar.com&#x2F;personalized-software-really-is-coming...</a>

As the poet wrote:I prefer MapQuestthat&#x27;s a good one, tooGoogle Maps is the besttrue thatdouble true!

I think &quot;Kagi Code&quot; or whatever it&#x27;s called is using Claude

Yah Claude tends to output 1200+ line architectural specification documents while Gemini tends to output ~600 line. (I just had to write 100+ architectural spec documents for 100+ different apps)Not sure why Claude is more thorough and complete than the other models, but it&#x27;s my go-to model for large projects.The OpenAI model outputs are always the smallest - 500 lines or so. Not very good at larger projects, but perfectly fine for small fixes.

This is exactly my approach. Use Gemini to come up with analysis and a plan, Claude to implement.

Much like others, this is my stack (or o1-pro instead of Gemini 2.5 Pro). This is a big reason why I use aider for large projects. It allows me to effortlessly combine architecture models and code writing models.I know in Cursor and others I can just switch models between chats, but it doesn&#x27;t feel intentional the way aider does. You chat in architecture mode, then execute in code mode.

I have been very brand loyal to claude also but the new gemini model is amazing and I have been using it exclusively for all of my coding for the last week.I am excited to try out this new model. I actually want to stay brand loyal to antropic because I like the people and the values they express.

Gemini is _very_ good at architecture level thinking and implementation.I tend to find that I use Gemini for the first pass, then switch to Claude for the actual line-by-line details.Claude is also far superior at writing specs than Gemini.

Idk I found Gemini 2.5 Breaking code style too often and introducing unneeded complexity on the top of leaving unfinished functions.

Gemini 2.5 Pro replaced Claude 3.7 for me after using nothing but claude for a very long time. It&#x27;s really fast, and really accurate. I can&#x27;t wait to try Claude 4, it&#x27;s always been the most &quot;human&quot; model in my opinion.

What&#x27;s the best way to use gemini? I&#x27;m currently pretty happy &#x2F; impressed with claude code via the CLI, its the best AI coding tool I&#x27;ve tried so far

I also recommend trying out Gemini, I&#x27;m really impressed by the latest 2.5. Let&#x27;s see if Claude 4 makes me switch back.

i was the same, but then slowly converted to Gemini. Still not sure how that happened tbh

It’s possible to get to know the quirks of these models and intuit what will and won’t work, and how to overcome those limitations. It’s also possible to just get to know, like, and trust their voice. I’m certain that brand awareness is also a factor for me in preferring Claude over ChatGPT etc

I wouldn&#x27;t go as far, but I actually have some loyalty to Claude as well. Don&#x27;t even know why, as I think the differences are marginal.

Something I’ve found true of Claude, but not other models, is that when the benchmarks are better, the real world performance is better. This makes me trust them a lot more and keeps me coming back.

I think it really depends on how you use it. Are you using an agent with it, or the chat directly?I&#x27;ve been pretty disappointed with Cursor and all the supported models. Sometimes it can be pretty good and convenient, because it&#x27;s right there in the editor, but it can also get stuck on very dumb stuff and re-trying the same strategies over and over againI&#x27;ve had really good experiences with o4-high-mini directly on the chat. It&#x27;s annoying going back and forth copying&#x2F;pasting code between editor and the browser, but it also keeps me more in control about the actions and the contextWould really like to know more about your experience

I&#x27;m slutty. I tend to use all four at once: Claude, Grok, Gemini and OpenAI.They keep leap-frogging each other. My preference has been the output from Gemini these last few weeks. Going to check out Claude now.

I&#x27;ve found myself having brand loyalty to Claude. I don&#x27;t really trust any of the other models with coding, the only one I even let close to my work is Claude. And this is after trying most of them. Looking forward to trying 4.

&gt; Extended thinking with tool use (beta): Both models can use tools—like web search—during extended thinking, allowing Claude to alternate between reasoning and tool use to improve responses.I&#x27;m happy that tool use during extended thinking is now a thing in Claude as well, from my experience with CoT models that was the one trick(tm) that massively improves on issues like hallucination&#x2F;outdated libraries&#x2F;useless thinking before tool use, e.g.o3 with search actually returned solid results, browsing the web as like how i&#x27;d do it, and i was thoroughly impressed – will see how Claude goes.

Booking visits to the dentist, hairdresser, any other type of service, renewing my phone or internet subscription at the lowest price, doing administrative tasks, adding the free game of the week on Epic Games Store to my library, finding the right houses to visit, etc.Basically anything that some startup has tried and failed at uberizing.

Why would coding be that unwanted task if one decided to work as a programmer? People&#x27;s unwanted tasks are cleaning the house, doing taxes etc.

What if coding is that unwanted task? Also, what are the tasks you are referring to, specifically?

I can&#x27;t think of more boring than marginal improvements on coding tasks to be honest.I want GenAI to become better at tasks that I don&#x27;t want to do, to reduce the unwanted noise from my life. This is when I&#x27;ll pay for it, not when they found a new way to cheat a bit more the benchmarks.At work I own the development of a tool that is using GenAI, so of course a new better model will be beneficial, especially because we do use Claude models, but it&#x27;s still not exciting or interesting in the slightest.

When will structured output be available? Is it difficult for anthropic because custom sampling breaks their safety tools?

Anthropic might be scammers. Unclear. I canceled my subscription with them months ago after they reduced capabilities for pro users and I found out months later that they never actually canceled it. They have been ignoring all of my support requests.. seems like a huge money grab to me because they know that they&#x27;re being out competed and missed the ball on monetizing earlier.

Claude register need a phone number, but cannot select China (+86), and even if I have a account, it may hard to purchase because the credit card issue.Some app like Windsurf can easily pay with Alipay, a everyone-app in China.

what do you mean purchasable from china? As in you are based in china or is there a way to game the tokens pricing

VSCode is not open source. It is proprietary. code-oss is however an MIT-licensed project. VSCodium is also an MIT-licensed project.

I don&#x27;t see any benefit in those VC funded wrappers over open source VS Code (or better, VSCodium) extensions like Roo&#x2F;Cline.They survive through VC funding, marketing, and inertia, I suppose.

How long will the VScode wrapper (cursor, windsurf) survive?Love to try the Claude Code VScode extension if the price is right and purchase-able from China.

&gt; definitely random which of these models can solve various problems with the same prompt.Yeah, this is borderline my feeling too. Kicking off Codex with the same prompt but four times sometimes leads to for very different but confident solutions. Same when using the chat interfaces, although it seems like Sonnet 3.7 with thinking and o1 Pro Mode is a lot more consistent than any Gemini model I&#x27;ve tried.

I just had some issue with RLS&#x2F;schema&#x2F;postgres stuff.  Gemini 2.5 Pro swung and missed, and talked a lot with little code, Claude Sonnet 4 solved. O1 Pro Solved.  It&#x27;s definitely random which of these models can solve various problems with the same prompt.

Anyone with access who could compare the new models with say O1 Pro Mode? Doesn&#x27;t have to be a very scientific comparison, just some first impressions&#x2F;thoughts compared to the current SOTA.

It feels like LLM progress in general has kinda stalled and we&#x27;re only getting small incremental improvements from here.I think we&#x27;ve reached peak LLM - if AGI is a thing, it won&#x27;t be through this architecture.

It feels like the days of Claude 2 -&gt; 3 or GPT 2-&gt;3 level changes for the leading models are over and you&#x27;re either going to end up with really awkward version numbers or just embrace it and increment the number. Nobody cares a Chrome update gives a major version change of 136-&gt;137 instead of 12.4.2.33 -&gt; 12.4.3.0 for similar kinds of &quot;the version number doesn&#x27;t always have to represent the amount of work&#x2F;improvement compared to the previous&quot; reasoning.

The improvement from Claude 3.7 wasn&#x27;t particularly huge. The improvement from Claude 3, however, was.

Slight decrease from Sonnet 3.7 in a few areas even. As always benchmarks say one thing, will need some practice with it to get a subjective opinion.

Benchmarks don&#x27;t tell you as much as the actual coding vibe though

beyond 4 thats no longer true - marketing took over from the research

My understanding for the original OpenAI and anthropic labels was essentially: gpt2 was 100x more compute than gpt1. Same for 2 to 3. Same for 3 to 4. Thus, gpt 4.5 was 10x more compute^If anthropic is doing the same thing, then 3.5 would be 10x more compute vs 3. 3.7 might be 3x more than 3.5. and 4 might be another ~3x.^ I think this maybe involves words like &quot;effective compute&quot;, so yeah it might not be a full pretrain but it might be! If you used 10x more compute that could mean doubling the amount used on pretraining and then using 8x compute on post or some other distribution

as soon as you publish a benchmark like this, it becomes worthless because it can be included in the training corpus

&gt; although it&#x27;s possible Claude 4 was trained on that discussion lolAlmost guaranteed, especially since HN tends to be popular in tech circles, and also trivial to scrape the entire thing in a couple of hours via the Algolia API.Recommendation for the future: keep your benchmarks&#x2F;evaluations private, as otherwise they&#x27;re basically useless as more models get published that are trained on your data. This is what I do, and usually I don&#x27;t see the &quot;huge improvements&quot; as other public benchmarks seems to indicate when new models appear.

I find it weird that it does a inner check on &#x27; num &gt; 99999&#x27;, which pretty much only checks for 100,000. It could check for 99993, but I doubt even that check makes it much faster.But have you checked with some other number than 30? Does it screw up the upper and lower bounds?

I did a quick review of its final answer and looks like there are logic errors.All three of them get the incorrect max-value bound (even with comments saying 9+9+9+9+3 = 30), so early termination wouldn&#x27;t happen in the second and third solution, but that&#x27;s an optimization detail. The first version would, however, early terminate on the first occurrence of 3999 and take whatever the max value was up to that point. So, for many inputs the first one (via solve_digit_sum_difference) is just wrong.The second implementation (solve_optimized, not a great name either) and third implementation, at least appear to be correct... but that pydoc and the comments in general are atrocious. In a review I would ask these to be reworded and would only expect juniors to even include anything similar in a pull request.I&#x27;m impressed that it&#x27;s able to pick a good line of reasoning, and even if it&#x27;s wrong about the optimizations it did give a working answer... but in the body of the response and in the code comments it clearly doesn&#x27;t understand digit extraction per se, despite parroting code about it. I suspect you&#x27;re right that the model has seen the problem solution before, and is possibly overfitting.Not bad, but I wouldn&#x27;t say it crushed it, and wouldn&#x27;t accept any of its micro-optimizations without benchmark results, or at least a benchmark test that I could then run.Have you tried the same question with other sums besides 30?

Might just be missing something, but isn&#x27;t 9+9+9+9+3=39? The largest number I believe is 99930? Also, it could further optimize by terminating digit sum calculations earlier if sum goes above 30 or could not reach 30 (num digits remaining * 9 is less than 30 - current_sum). imo this is pretty far from &quot;crushing it&quot;

&gt; although it&#x27;s possible Claude 4 was trained on that discussion lolThis is why we can&#x27;t have consistent benchmarks

The first iteration vectorized with numpy is the best solution imho. The only additional optimization is using modulo 9 to give you a sum of digits mod 9; that should filter out approximately 1&#x2F;9th of numbers. The digit summing is the slow part so reducing the number of values there results in a large speedup. Numpy can do that filter pretty fast as `arr = arr[arr%9==3]`With that optimization its about 3 times faster, and all of the none numpy solutions are slower than the numpy one. In python it almost never makes sense to try to manually iterate for speed.

Same for me, with this past year&#x27;s Advent of Code. All the models until now have been stumped by Day 17 part 2. But Opus 4 finally got it! Good chance some of that is in its training data, though.

So I decided to try Claude 4 Sonnet against my &quot;Given a list of 1 million random integers between 1 and 100,000, find the difference between the smallest and the largest numbers whose digits sum up to 30.&quot; benchmark I tested against Claude 3.5 Sonnet: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=42584400">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=42584400</a>The results are here (<a href="https:&#x2F;&#x2F;gist.github.com&#x2F;minimaxir&#x2F;1bad26f0f000562b1418754d67a8161b" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;minimaxir&#x2F;1bad26f0f000562b1418754d67...</a> ) and it utterly crushed the problem with the relevant microoptimizations commented in that HN discussion (oddly in the second pass it a) regresses from a vectorized approach to a linear approach and b) generates and iterates on three different iterations instead of one final iteration), although it&#x27;s possible Claude 4 was trained on that discussion lol.EDIT: &quot;utterly crushed&quot; may have been hyperbole.

They say in the blog post that tool use has improved dramatically: parallel tool use, ability to use tools during thinking and more.

Hey, at least they incremented the version number.  I&#x27;ll take it.

To be fair, a lot of people said 3.7 should have just been called 4. Maybe they&#x27;re just bridging the gap.

I was about to comment on a past remark from Anthropic that the whole reason for the convoluted naming scheme was because they wanted to wait until they had a model worth of the &quot;Claude 4&quot; title.But because of all the incremental improvements since then, the irony is that this merely feels like an incremental improvement. It obviously is a huge leap when you consider that the best Claude 3 ever got on SWE-verified was just under 20% (combined with SWE-agent), but compared to Claude 3.7 it doesn&#x27;t feel like that big of a deal, at least when it comes to SWE-bench results.Is it worthy? Sure, why not, compared to the original Claude 3 at any rate, but this habit of incremental improvement means that a major new release feels kind of ordinary.

Is this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn&#x27;t generalize to intelligence. The agentic tooluse didn&#x27;t improve and this says to me that it&#x27;s not really smarter.

Each model has their own strength and weaknesses tho. You really shouldn’t be using one model for everything. Like, Claude is great at coding but is expensive so you wouldn’t use them for debugging to writing test benches. But the OpenAI models suck at architecture but are cheap, so are ideal for test benches, for example.

Isn&#x27;t that an argument to jump around? Since performance improves so rapidly between models

You should at least have two to sanity check difficult programming solutions.

A man with one compass knows where he&#x27;s going; a man with two compasses is never sure.

My advice: don&#x27;t jump around between LLMs for a given project. The AI space is progressing too rapidly right now. Save yourself the sanity.

Vibe code an eval harness with a web dashboard

&gt; How important is it to be using SOTA?I believe in using the best model for each use case. Since I’m paying for it, I like to find out which model is the best bang for my buck.The problem is that, even when comparing models according to different use cases, better models eventually appear, and the models one uses eventually change as well — for better or worse. This means that using the same model over and over doesn’t seem like a good decision.

How important is it to be using SOTA? Or even jump on it already?Feels a bit like when it was a new frontend framework every week. Didn&#x27;t jump on any then. Sure, when React was the winner, I had a few months less experience than those who bet on the correct horse. But nothing I couldn&#x27;t quickly catch up to.

I&#x27;d love something like litellm, but simpler. I&#x27;m not provisioning models for my organization, I don&#x27;t need to granularly track spend, I just want one endpoint to point every tool or client at for ease of configuration and curiosity around usage.

Have you tried a package like LiteLLM so that you can more easily validate and switch to a newer model?The key seems to be in curating your application&#x27;s evaluation set.

It’s been hard to keep up with the evolution in LLMs. SOTA models basically change every other week, and each of them has its own quirks.Differences in features, personality, output formatting, UI, safety filters… make it nearly impossible to migrate workflows between distinct LLMs. Even models of the same family exhibit strikingly different behaviors in response to the same prompt.Still, having to find each model’s strengths and weaknesses on my own is certainly much better than not seeing any progress in the field. I just hope that, eventually, LLM providers converge on a similar set of features and behaviors for their models.

This is starting to get ridiculous. I am busy with life and have hundreds of tabs unread including one [1] about Claude 3.7 Sonnet and Claude Code and Gemini 2.5 Pro. And before any of that Claude 4 is out. And all the stuff Google announced during IO yday.So will Claude 4.5 come out in a few months and 5.0 before the end of the year?At this point is it even worth following anything about AI &#x2F; LLM?[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=43163011">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=43163011</a>

Anyone know when the o4-x-mini release is being announced? I thought it was today

Anyone have a link to the actual Anthropic official vscode extension? Struggling to find it.edit: run `claude` in a vscode terminal and it will get installed. but the actual extension id is `Anthropic.claude-code`

Here&#x27;s an example non-power-user usecase for CoT:Sometimes when I miss to specify a detail in my prompt and it&#x27;s just a short task where I don&#x27;t bother with long processes like &quot;ask clarifying questions, make a plan and then follow it&quot; etc etc, I see it talking about making that assumption in the CoT and I immediately cancel the request and edit the detail in.

You&#x27;re accusing me of mind reading other users, but then proceed to engage in your own mind reading of those same users.Have a look in Gemini related subreddits after they nerfed their CoT yesterday. There&#x27;s nobody happy about this trend. A high quality CoT that gets put through a small LLM is really no better than noise. Paternalistic noise. It&#x27;s not worth reading. Just don&#x27;t even show me the CoT at all.If someone is paying for Opus 4 then they likely are a power user, anyway. They&#x27;re doing it for the frontier performance and I would assume such users would appreciate the real CoT.

If _you_ really believe this is what all users want, _you_ should reconsider. Your describing a feature for power users. It should be a toggle but it&#x27;s silly to say it didn&#x27;t improve UX for people who don&#x27;t want to read strange babbling chains of thought.

When you do a chat are reasoning traces for prior model outputs in the LLM context?

&gt; I don&#x27;t think this is referring to real-time summarizing for the users&#x27; sake.That&#x27;s exactly what it&#x27;s referring to.

I believe this is to improve performance by shortening the context window for long thinking processes. I don&#x27;t think this is referring to real-time summarizing for the users&#x27; sake.

<pre><code> &gt; Finally, we&#x27;ve introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full.
</code></pre>
This is not better for the user. No users want this. If you&#x27;re doing this to prevent competitors training on your thought traces then fine. But if you really believe this is what users want, you need to reconsider.