Colleagues were saying that horizon alpha and beta were looking better than claude4 for frontend stuff, especially newer frameworks. I think the idea of having full + mini + nano is really good, as long as the smaller ones can reasonably handle small-ish tasks. You&#x27;d have your architect &#x2F; plan whatever sessions with the large one, scoping out regular tasks for the -mini version and then the really easy ones to -nano.4.1 was almost usable in that fashion. I had 4.1-nano working in cline with really trivial stuff (add logging, take this example and adapt it in this file, etc) and it worked pretty well most of the time.

That&#x27;s been my experience too. Even though Gemini also does seem to do the fancy one-shot demo code well, in day to day coding, Claude seems to do a much better job of just understanding how programming actually works, what to do, what not to do, etc.

Agreed, I always give my one pager product briefs to AI to break down into phases and tasks, and then progress trackers. I explicitly prompt for verbose phases, tasks and test plans.Yesterday without much promoting Claude 4.1 gave me 10 phases, each with 5-12 tasks that could genuinely be used to kanban out a product step by step.Claude 3.7 sonnet was effectively the same with fewer granular suggestions for programming strategies.Gemini 2.5 gave me a one pager back with some trivial bullet points in 3 phases, no tasks at all.o3 did the same as as Gemini, just less coherent.Claude just has whatever the thing is for now

Claude is fast too, Gemini isn’t as good and just gets hung up on things Claude doesn’t.

The secret is just better context engineering. There is no other “secret” sauce, all these models are built on the same concepts.

My experience has been that Claude Code is exceptional at tool use (and thus working with agentic IDEs) but... not the smartest coder. It will happy re-invent the wheel, create silos, or generate terrible code that you&#x27;ll only discover weeks or months later. I&#x27;ve had to rollback weeks of code to discover major edge regressions that Claude had introduced.Now, someone will say &#x27;add more tests&#x27;. Sure. But that&#x27;s a bandaid.I find that the &#x27;smarter&#x27; models like Gemini and o3 output better quality code overall and if you can afford to send them the entire context in a non-agentic way .. then they&#x27;ll generate something dramatically superior to the agentic code artifacts.That said, sometimes you just want speed to proof a concept and Claude is exceptional there. Unfortunately, proof of concepts often... become productionized rather than developers taking a step back to &quot;do it right&quot;.

Whatever the benchmarks might say, there&#x27;s something about Claude that seems to deliver consistently (although not always perfect) quite reliable outputs across various coding tasks. I wonder what that &#x27;secret sauce&#x27; might be and whether GPT-5 has figured it out too.

Claude Code is good because the Anthropic models are trained&#x2F;finetuned to be good at using it.

Well, since (like you pointed out) using the Anthropic models in different settings is not that exciting anymore, the difference is what Claude Code does. It&#x27;s a good product.

Refactors, building non-trivial features (you can first write out a spec and have it follow that), understanding my code, writing tests, writing good quality documentation. Reasoning about my existing data model and where to plug into it.On and on and on. Coming up with test plans, edge cases, accounting for the edge cases in its programming. Programming defensively. Fixing bugs.

Killing it - at what type of coding task? What &quot;bigger advantages&quot; specifically? What is night and day?

Yup, Claude has been kicking GPT&#x27;s ass for months now

For day to day coding, I&#x27;ve found Anthropic to be killing it with Sonnet 3.7 and now Sonnet 4, and Claude Code feeling like it has even bigger advantages over when it&#x27;s used in Cursor (And I can&#x27;t explain why).I don&#x27;t even try to use the OpenAI models because it&#x27;s felt like night and day.Hopefully GPT-5 helps them catch up. Although I&#x27;m sure there are 100 people that have their own personal &quot;hopefully GPT-5 fixes my personal issue with GPT4&quot;

It&#x27;s very doubtful that they&#x27;d have any kind of magical breakthrough that makes the model anything other than incrementally better right now.

I&#x27;m guessing they realized they have rip off the bandaid and release a GPT 5 at some point, and we&#x27;re gonna see a relatively incremental improvement.

It means nothing now.It&#x27;s the same as 4G vs 5G. They have a technical definition, but it&#x27;s all about marketing.

It means 5 is more than 4, Claude only has a 4. Clearly 5 is better

GPT-5 should mean a brand new model&#x2F;architecture trained from scratch.

The leak last night seems to indicate this will be coding focused.I&#x27;d imagine this must be a big leg up on Anthropic to warrant the &quot;GPT-5&quot; name?

Why do you hope this so much? Any personal reasons?

They are the worst affected. Nothing you can do about it.

So, arbitrarily, it will just be “JavaScript&#x2F;TypeScript developers” affected and everyone else will be fine?

Tools like GPT-5 will transform web development rather than replace developers - the most valuable skills will shift toward problem definition, architecture design, and quality verification while repetitive coding gets automated.

damn did a front end engineer hook up with your wife? What did I do to you?

I would qualify that by saying developers who do not have Product Owner skills and Product Owners who do not have developer skills will be made obsolete.Having both eliminates a feedback loop and the LLM enables you to get shit done fast.

I hope that this live stream will tell you that this will be the definitive reason why web developers, JavaScript &#x2F; TypeScript developers are going to be made completely obsolete at worse and at best, their jobs will be reduced at all levels.The best part is, this is not even the real definition of &quot;AGI&quot; yet (whatever that means at this point).More like 10% of the capability that was promised and already the flow of capital from the inflated salaries of the past decade are going to the top AI researchers.

Yep, agreed -- the repo is talking about &#x27;one prompt with an agentic coding platform, but... at least here there&#x27;s nothing particularly new.Will be interesting to see what pushing it harder does – what the new ceiling is. 88% on aider polyglot is pretty good!

These are honestly pretty disappointing :&#x2F; this quality was possible with Claude Code months ago

Yeah, I guess it&#x27;s just the easiest thing to generate and evaluate.A more useful demonstration like making large meaningful changes to a large complicated codebase would be much harder to evaluate since you need to be familiar with the existing system to evaluate the quality of the transformation.Would be kinda cool to instead see diffs of nontrivial patches to the Ruby on Rails codebase or something.

&gt; Honestly, why would anyone find this information useful?This seems to impress the mgmt types a lot, e.g. &quot;I made a WHOLE APP!&quot;, when basically what most of this is is frameworks and tech that had crappy bootstrapping to begin with (React and JS are rife with this, in spite of their popularity).

Honestly, why would anyone find this information useful? Creating a brand new greenfield project is a terrible test. Because literally anything it outputs as long as it looks good as long as it works following the happy path. Coding with LLMs falls apart in situations where complex reasoning is required. Situations such as having debugging issues in a service where there&#x27;s either no framework in use or they&#x27;ve significantly modified a framework to make it better suit the authors needs.

&gt; the internet is full of JS code and at this point I expect them to be good at it.Isn&#x27;t that the rub though? It&#x27;s not an ex nihlo &quot;intelligence&quot;, it&#x27;s whatever stuff it&#x27;s trained on and can derive completions from.

&gt; I wish they wouldn&#x27;t use JS to demonstrate the AI&#x27;s coding abilities - the internet is full of JS code and at this point I expect them to be good at it. Show me examples in complex (for lack of a better word) languages to impress me.Agreed. The models break down on not even that complex of code either, if it&#x27;s not web&#x2F;javascript. Was playing with Gemini CLI the other day and had it try to make a simple Avalonia GUI app in C#&#x2F;.NET, kept going around in circles and couldn&#x27;t even get a basic starter project to build so I can imagine how much it&#x27;d struggle with OCaml or other more &quot;obscure&quot; languages.This makes the tech even less useful where it&#x27;d be most helpful - on internal, legacy codebases, enterprisey stuff, stacks that don&#x27;t have numerous examples on github to train from.

I&#x27;ve tried with many models to program in mathematica and sagemath; they&#x27;re terrible, even with lots of hints.

I used ChatGPT to convert an old piece of OCaml code of mine to Rust and while it didn&#x27;t really work—and I didn&#x27;t expect it to—it seemed a very reasonable starting point to actually do the rest of the work manually.

The snake game they showcased - if you ask Qwen3-coder-30b to generate a snake game in JS - it generates the exact same layout,  the exact same two buttons below, and the exact same text under the 2 buttons. It just regurgigates its training data.

I wish they wouldn&#x27;t use JS to demonstrate the AI&#x27;s coding abilities - the internet is full of JS code and at this point I expect them to be good at it. 
Show me examples in complex (for lack of a better word) languages to impress me.I recently used OpenAI models to generate OCaml code, and it was eye opening how much even reasoning models are still just copy and paste machines. 
The code was full of syntax errors, and they clearly lacked a basic understanding of what functions are in the stdlib vs those from popular (in OCaml terms) libraries.Maybe GPT-5 is the great leap and I&#x27;ll have to eat my words, but this experience really made me more pessimistic about AI&#x27;s potential and the future of programming in general.
I&#x27;m hoping that in 10 years niche languages are still a thing, and the world doesn&#x27;t converge toward writing everything in JS just because AIs make it easier to work with.

For those who haven&#x27;t seen, a little bit of early stuff:Official OpenAI gpt-5 coding examples repo:
<a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples</a> (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44826439">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44826439</a>)Github leak: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44826439">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44826439</a>

It will be like coming home after such a long time using Sonnet 4 for all code and UI&#x2F;UX work. I do hope sincerely this brings OpenAI back on top! Would be awesome to have a new king again.&quot;This repository contains a curated collection of demo applications generated entirely in a single GPT-5 prompt, without writing any code by hand.&quot;<a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples</a>This is promising!

I don&#x27;t think YouTube comment section ever contain useful information regardless of what the video&#x2F;stream is about.

I don’t know. Live comment feeds on popular streams makes me question democracy.

It&#x27;s a shame, because that seems like the sort of thing LLMs would be able to moderate quite effectively, if YouTube was willing to put the effort in.

Have you seen YouTube comments on videos like this? It&#x27;s all-crypto scams, bots responding to other bots, and occasional racism.

What&#x27;s up with their very first eval? The SWE bars and numbers don&#x27;t line up.

Assuming even 10% of YouTube commenters are real people.

&gt; comments turned offyikes - the poor executive leadership’s fragile egos cannot take the criticism.

Honestly, you&#x27;re probably right. It&#x27;s quickly become a pretty weak eval, but the guy that&#x27;s running that eval is excellent. I&#x27;d much rather the evals people were using to test these things looked more like classic&#x2F;boring engineering problems: deploy to dev&#x2F;test&#x2F;stage&#x2F;prod with digital ocean, cloudflare, github, and a common git flow. Boring problem, I know, but that problem is wildly complex when you start to add a few extra dimensions (frontend vs backend, ports shifting between deployments, local deployments, etc.).

i think the point is people assume models arent overfitting for it, and its a fun&#x2F;silly way to potentially gauge its general abilities

&gt; That problem along with its many solutions are surely littered throughout the training data. Not to mention, it would be trivial to overfit on that problem.It would be trivial to over-fit, if that was their goal.But why would there be a large number of good SVG images of pelicans on bikes? Especially relative to all the things we actually want them to generalise over?Surely most of the SVG images of pelicans on bikes are, right now, going to be &quot;look at this rubbish AI output&quot;? (Which may or may not be followed by a comment linking to that artist who got humans to draw bikes and oh boy were those humans wildly bad at drawing bikes, so an AI learning to draw SVGs from those bitmap pictures would likely also still suck…)

Maybe we can try “dog in a paraglider”? If it fails then we know it’s overfitting, if it works then the model generalises well?

That problem along with its many solutions are surely littered throughout the training data. Not to mention, it would be trivial to overfit on that problem. I don&#x27;t know why people still reference that.

It&#x27;s very unclear if OpenAI has been casually leaking things to create buzz, but a few days ago there was a pretty stunning pelican on a bike attempt: <a href="https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;OpenAI&#x2F;comments&#x2F;1mettre&#x2F;gpt5_is_already_ostensibly_available_via_api&#x2F;n6cekdr&#x2F;" rel="nofollow">https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;OpenAI&#x2F;comments&#x2F;1mettre&#x2F;gpt5_is_alr...</a>In practice, it&#x27;s very clear to me that the most important value in writing software with an LLM isn&#x27;t it&#x27;s ability to one-shot hard problems, but rather it&#x27;s ability to effectively manage complex context. There are no good evals for this kind of problem, but that&#x27;s what I&#x27;m keenly interested in understanding. Show me GPT-5 can move through 10 steps in a list of tasks without completely losing the objective by the end.

Since the stream has been on some starting screen for several minutes, I went to check whether there are watch-along streams on Twitch for this - there are a few, and for some reason every one of them is in Spanish. I know Spanish-language streams are a big thing, but it&#x27;s curious that there&#x27;s three Spanish GPT-5 watchalong streams (two with 50-ish viewers and one with 2.5k) and none in English.edit: YouTube has a few English &quot;watch party&quot; streams, although there too, the Spanish ones have many times more viewers.

They do know when it starts. They have it prerecorded and start it at a specific time. This one started 10 minutes before.

A bit unrelated: The &quot;countdown animation&quot;, just like Google I&#x2F;O&#x27;s, how do people make those? The countdown is probably dynamically generated, as they don&#x27;t know when the event will actually start? Is there like a JavaScript library, or CapCut template, or something?Especially Google IO, each year is different, it seems purpose built?

Did you win the race to be the first comment?

Congratulations on winning the race to post the announcement :)

<a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples</a>

Aaand hugged to death.edit:livestream here: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;live&#x2F;0Uu_VJeVVfo" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;live&#x2F;0Uu_VJeVVfo</a>

Was just skimming along that review, while watching the live-stream, where they just mentioned how much better at writing prose GPT-5 is, while I skimmed across:&gt; It’s actually worse at writing than GPT-4.5Sounds like we need to wait a bit for the dust to settle before one can trust anything one hears&#x2F;reads :)

Great writeup. I particularly like the idea of splitting your tools into the four buckets.1)Internal Retrieval2)Web Search3)Code Interpreter4)ActionsHow did you come up with this idea?

”I think GPT-5 is the closest to AGI we’ve ever been”Sorry, but this sounds like overly sensational marketing speak and just leaves a bad taste in the mouth for me.

Hi Swyx I always appreciate your insights, something you wrote really resonated with a personal theory I&#x27;ve been developing:&gt;&quot;While I never use AI for personal writing (because I have a strong belief in writing to think)&quot;The optimal AI productivity process is starting to look like:AI Generates &gt; Human Validates &gt; LoopYet cognitive generation is how humans learn and develop cognitive strength, as well as how they maintain such strength.Similar to how physical activity is how muscles&#x2F;bone density&#x2F;etc grow, and how body tissues maintain.Physical technology freed us from hard physical labor that kept our bodies in shape -- at a cost of physical atrophy.AI seems to have a similar effect for our minds. AI will accelerate our cognitive productivity, and allow for cognitive convenience -- at a cost of cognitive atrophy.At present we must be intentional about building&#x2F;maintaining physical strength (dedicated strength training, cardio, etc).Soon we will need to be intentional about building&#x2F;maintaining cognitive strength.I suspect the workday&#x2F;week of the future will be split on AI-on-a-leash work for optimal productivity, with carve-outs for dedicated AI-enhanced-learning solely for building&#x2F;maintaining cognitive health (where productivity is not the goal, building&#x2F;maintaining cognition is). Similar to how we carve out time for working out.What are your thoughts on this? Based on what you wrote above, it seems you have similar feelings?Is there a name for this theory?If not can you coin one? You&#x27;re great at that :)

Re the substantive example, doesn&#x27;t Claude Code already do this? When I&#x27;m vibe-coding scripts or even mobile apps cc can be pretty aggressive about adding targeted logging to solve specific issues.

our hands on review: <a href="https:&#x2F;&#x2F;www.latent.space&#x2F;p&#x2F;gpt-5-review" rel="nofollow">https:&#x2F;&#x2F;www.latent.space&#x2F;p&#x2F;gpt-5-review</a>basically in my testing really felt that gpt5 was &quot;using tools to think&quot; rather than just &quot;using tools&quot;. it gets very powerful when coding long horizon tasks (a separate post i&#x27;m publishing later).to give one substantive example, in my developer beta (they will release the video in a bit) i put it to a task that claude code had been stuck on for the last week - same prompts - and it just added logging to instrument some of the failures that we were seeing and - from the logs that it added and asked me to rerun - figured out the solve.

Which graph? There are dozens of graph on the page.

Holy misleading graph Batman&#x2F;Altman!Academic benchmark score improves only 5% but they make the bar 50% higher.

<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=0Uu_VJeVVfo" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=0Uu_VJeVVfo</a>

# GPT5 all official linksLivestream link: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;live&#x2F;0Uu_VJeVVfo" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;live&#x2F;0Uu_VJeVVfo</a>Research blog post: <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-5&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-5&#x2F;</a>Developer blog post: <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-5-for-developers" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-5-for-developers</a>API Docs: <a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;guides&#x2F;latest-model" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;guides&#x2F;latest-model</a>Note the free form function calling documentation: <a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;guides&#x2F;function-calling#context-free-grammars" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;guides&#x2F;function-calling#con...</a>GPT5 prompting guide: <a href="https:&#x2F;&#x2F;cookbook.openai.com&#x2F;examples&#x2F;gpt-5&#x2F;gpt-5_prompting_guide" rel="nofollow">https:&#x2F;&#x2F;cookbook.openai.com&#x2F;examples&#x2F;gpt-5&#x2F;gpt-5_prompting_g...</a>GPT5 new params and tools: <a href="https:&#x2F;&#x2F;cookbook.openai.com&#x2F;examples&#x2F;gpt-5&#x2F;gpt-5_new_params_and_tools" rel="nofollow">https:&#x2F;&#x2F;cookbook.openai.com&#x2F;examples&#x2F;gpt-5&#x2F;gpt-5_new_params_...</a>GPT5 frontend cookbook: <a href="https:&#x2F;&#x2F;cookbook.openai.com&#x2F;examples&#x2F;gpt-5&#x2F;gpt-5_frontend" rel="nofollow">https:&#x2F;&#x2F;cookbook.openai.com&#x2F;examples&#x2F;gpt-5&#x2F;gpt-5_frontend</a>prompt migrator&#x2F;optimizor <a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;chat&#x2F;edit?optimize=true" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;chat&#x2F;edit?optimize=true</a>Enterprise blog post: <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;gpt-5-new-era-of-work" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;gpt-5-new-era-of-work</a>System Card: <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;gpt-5-system-card&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;gpt-5-system-card&#x2F;</a>What would you say if you could talk to a future OpenAI model? <a href="https:&#x2F;&#x2F;progress.openai.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;progress.openai.com&#x2F;</a>coding examples: <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-5-coding-examples</a>

Wished this version would be called OpenAI-GPT-25.8

The free version of Gemini 2.5 mini is great for this- doesn&#x27;t need a transcript, apparently can analyse the video as well

Considering how they hyped it up (eg. “Lol normies go about their day and have no idea whats coming etc”) they have to show some AGI level llm or stop overhyping their 2% improvements.

&gt; there are alternative LLMs and services that both outperform and are a fraction of OpenAI&#x27;s pricingLike what? Deepseek?

Hopefully, OpenAI makes their APIs more affordable. So far, there are alternative LLMs and services that both outperform and are a fraction of OpenAI&#x27;s pricing. OpenAI is usually one of (if not) the most expensive option, maybe that&#x27;s because of the brand identification. Not really sure why people pay that premium.

Thanks for pointing that out, I missed that. Very curious how they&#x27;ll measure that. Given they&#x27;re in the double-digit billions of revenue I&#x27;d assume they can reason they&#x27;ll be there very soon.

Who knew we&#x27;ve had AGI for something like three hundred years? (Or, only had NGI for so long?)

That not their definition of AGI. 
It&#x27;s &quot;a highly autonomous system that can outperform humans at most economically valuable work.&quot;

OpenAI has clear and strange definition of AGI in contract with MSFT: it should produce 100B economic impact.

Got it. I should have stated Elon used to describe their latest model release, not coined. Thank you

Elon did not coin this, Kurzweil has been using this coinage for a lot longer.

After watching Claude cheerfully circle Team Rocket HQ for a month, I can attest that perseverence is not what stands between current models and PhDs.

If you&#x27;re ever in Austin, gimme a call. I&#x27;d love to be a dumb &amp; doctored, together.

As a fairly dumb person with a PhD, I can attest that a degree means perseverance, not intelligence.

As a non-PhD, most PhD&#x27;s insist most PhD&#x27;s aren&#x27;t smart - and yet I find them to generally be the smartest people I know :)

I gotta say PhDs are a dime a dozen these days, and yet we are talking about science stagnation.And PhDs are not very smart imho (I am one)

I think “PhD level knowledge” is probably a more meaningful and accurate phrase.

It&#x27;s very interesting how memetic the language around different models is. Elon seems to have coined &quot;PhD level intelligence in all topics&quot; and now Sam repeated it in his presentation. Despite it not having an actual meaning. I think OpenAI will coin they&#x27;ve achieved AGI first (as they have incentives to based on the rumored contract with MSFT), and then everyone will claim we&#x27;ve achieved it.

Neat, more scalable intelligence for me to tell &quot;plz fix&quot; over my code

This sounds like the kind of thing:1. I desperately want (especially from Google)2. Is impossible, because it will be super gamed, to the detriment of actually building flexible flows.

The eval bar I want to see here is simple: over a complex objective (e.g., deploy to prod using a git workflow), how many tasks can GPT-5 stay on track with before it falls off the train. Context is king and it&#x27;s the most obvious and glaring problem with current models.

Something was off. I think they had an expensive lighting setup with no one that knew what looked good. Everything was very diffused and flat. Like I would expect AI to replicate.

It&#x27;s only when he stumbled a bit that I could tell for sure (well, mostly) that it wasn&#x27;t an AI generated video - the corporate speak, body language mannerisms of Sam Altman, camera angles, all seemed pretty plausibly AI-generated!

&gt; Will wait for vibe check from simonw<a href="https:&#x2F;&#x2F;openai.com&#x2F;gpt-5&#x2F;?video=1108156668" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;gpt-5&#x2F;?video=1108156668</a>2:40 &quot;I do like how the pelican&#x27;s feet are on the pedals.&quot; &quot;That&#x27;s a rare detail that most of the other models I&#x27;ve tried this on have missed.&quot;4:12 &quot;The bicycle was flawless.&quot;5:30 Re generating documentation: &quot;It nailed it. It gave me the exact information I needed. It gave me full architectural overview. It was clearly very good at consuming a quarter million tokens of rust.&quot; &quot;My trust issues are beginning to fall away&quot;Edit: ohh he has blog post now: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44828264">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44828264</a>

&gt; Not much explanation yet why GPT-5 warrants a major version bumpExactly. Too many videos - too little real data &#x2F; benchmarks on the page. Will wait for vibe check from simonw and others

o3 pricing: $8&#x2F;Mtok outGPT-5 pricing: $10&#x2F;Mtok outWhat am I missing?

its &gt;o3 performance at gpt4 price. seems pretty obvious

Note GPT-5&#x27;s subtle mouthfeel reminiscent of cranberries with a touch of bourbon.

I can already see LLMs Sommeliers: Yes, the mouthfeel and punch of GPT-5 it&#x27;s comparable to the one of Grok 4, but it&#x27;s tenderness lacks the crunch from Gemini 2.5 Pro.

Informed audiophiles rely on Klippel output now

Come on, we aren&#x27;t even close to the level of audiophile nonsense like worrying about what cable sounds better.

Well, reduced sibilance is an ordinary and desirable thing. A better &quot;audiophile absurdity&quot; example would be $77,000 cables, freezing CDs to improve sound quality, using hospital-grade outlets, cryogenically frozen outlets (lol), the list goes on and on

You need to burn-in your LLM by using for 100 hours before you see the true performance of it.

Always have been. This LLM-centered AI boom has been my craziest and most frustrating social experiment, propped up by the rhetoric (with no evidence to back it up) that this time we finally have the keys to AGI (whatever the hell that means), and infused with enough AstroTurfing to drive the discourse into ideological stances devoid of any substance (you must either be a true believer or a naysayer). On the plus side, it appears that this hype train is taking a bump with GPT-5.

We’re at the audiophile stage of LLMs where people are talking about the improved soundstage, tonality, reduced sibilance etc

There&#x27;s a bunch of benchmarks on the intro page including AIME 2025 without tools, SWE-bench Verified, Aider Polyglot, MMMU, and HealthBench Hard (not familiar with this one): <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-5&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-5&#x2F;</a>Pretty par for course evals at launch setup.

Who said anything about sustainable? The only goal here is to hobble to the next VC round. And then the next, and the next, ...

It doesn&#x27;t even have that, knowledge cutoff is in 2024.

It has the last ~6 months worth of flavor of the month Javascript libraries in it&#x27;s training set now, so it&#x27;s &quot;better at coding&quot;.How is this sustainable.

I find it struggles to even refactor codebases that aren&#x27;t that large. If you have a somewhat complicated change that spans the full stack, and has some sort of wrinkle that makes it slightly more complicated than adding a data field, then even the most modern LLMs seem to trip on themselves. Even when I tell it to create a plan for implementation and write it to a markdown file and then step through those steps in a separate prompt.Not that it makes it useless, just that we seem to not &quot;be there&quot; yet for the standard tasks software engineers do every day.

Agree, I think they&#x27;ll need to move to performance now. If a model was comparable to Claude 4, but took like 500ms or less per edit. A quicker feedback loop would be a big improvement.

As someone who tries to push the limits of hard coding tasks (mainly refactoring old codebases) to LLMs with not much improvement since the last round of models, I&#x27;m finding that we are hitting the reduction of rate of improvement on the S-curve of quality. Obviously getting the same quality cheaper would be huge, but the quality of the output day to day isn&#x27;t noticeable to me.

Yeah. We&#x27;re entered the Smartphone stage: &quot;You want the new one because it&#x27;s the new one.&quot;

When they were about to release gpt4 I remember the hype was so high there were a lot of AGI debates. But then was quickly out-shadowed by more advanced models.People knew that gpt5 wouldn’t be an AGI or even close to that. It’s just an updated version. GptN would become more or leas like an annual release.

For fun, I asked it how much better it is than GPT-4. It started a rap battle against itself :P<a href="https:&#x2F;&#x2F;chatgpt.com&#x2F;share&#x2F;6895d5da-8884-8003-bf9d-1e191b11d3d1" rel="nofollow">https:&#x2F;&#x2F;chatgpt.com&#x2F;share&#x2F;6895d5da-8884-8003-bf9d-1e191b11d3...</a>

I didn&#x27;t think GPT-4 warranted a major version bump. I do not believe that Open AI&#x27;s benchmarks are legitimate and I don&#x27;t think they have been for quite some time, if ever.

I think the biggest tell for me was having the leader of Cursor up vouching for the model, who has been a big proponent of Claude in Cursor for the last year. Doesn&#x27;t seem like a light statement.

The marketing copy and the current livestream appear tautological: &quot;it&#x27;s better because it&#x27;s better.&quot;Not much explanation yet why GPT-5 warrants a major version bump. As usual, the model (and potentially OpenAI as a whole) will depend on output vibe checks.

SWE-Bench is also not stellar. &quot;It&#x27;s important to remember&quot; that:- they are only evals- this is mostly positioned as a general consumer product, they might have better stuff for us nerds in hand.

The Polyglot aider improvement over o3 is imperceptible, not great.

There&#x27;s no way to directly contact another user other than by replying to a post of theirs and hoping for the best.If you email us at hn@ycombinator.com and tell us who you want to contact, we might be able to email them and ask if they would be willing to have you contact them. No guarantees though!

At this point the prediction for SWE bench (85% by end of this month) is not materializing. We&#x27;re actually quite far away.

Short?It&#x27;s a perfect situation for Nvidia. You can see that after months of trying to squeeze out all % of marginal improvements, sama and co decided to brand this GPT-4.0.0.1 version as GPT-5. This is all happening on NVDA hardware, and they are gonna continue desperately iterating on tiny model efficiencies until all these valuation $$$ sweet sweet VC cash run out (most of it directly or indirectly going to NVDA).

It&#x27;s good for NVDA if the AI companies can&#x27;t squeeze more performance out of the same compute, which is the case if GPT-5 underperforms

I think one thing to look out for are &quot;deliberately&quot; slow models. We are currently using basically all models as if we needed them in an instant loop, but many of these applications do not have to run that fast.To tell a made-up anecdote: A colleague told me how his professor friend was running statistical models over night because the code was extremely unoptimized and needed 6+ hours to compute. He helped streamline the code and took it down to 30 minutes, which meant the professor could run it before breakfast instead.We are completely fine with giving a task to a Junior Dev for a couple of days and see what happens. Now we love the quick feedback of running Claude Max for a hundred bucks, but if we could run it for a buck over night? Would be quite fine for me as well.

Putting on my speculator hat here, it&#x27;s as much about psychology and crowd behavior as fundamentals. Probably wait till it drops 30% and the news has &quot;is it all over for AI?&quot; stories. It&#x27;ll then bounce back and then you sell the top of the bounce back.

When to short NVIDIA? I guess when chinese get their cards production

Good thing they didn&#x27;t nuke the data centers after all!

My hunch is generative pre-trained transformers aren&#x27;t going to do it just by scaling. Humans learn and modify their models as they go, it isn&#x27;t all pre-training and then fixed. We need a modified algorithm.The current situation is kind of like a grand prize where Zuck or similar will hand $1bn to anyone who cracks it. That&#x27;s a huge incentive for people to have a go.

Nothing about sentience is obvious.  If the trees were sentient, would it be obvious? Is it therefore obvious that they&#x27;re not?  I think its a no in both cases.  Same argument applies to AI model.

Obviously, they haven&#x27;t figured out anything remotely sentient. It&#x27;s cool as fuck, but it&#x27;s not actually thinking. Thinking requires learning. You could show it a cat and it would still tell you it&#x27;s a dog, no matter how many times you try and tell it.

But but but my tech bro CTO said grok IS AGI

Does this mean AGI is cancelled? 2027 hard takeoff was just sci-fi?

The current livestream listed the benchmarks (curiously comparing it only to previous GPT models and not competitors)

I wish they posted detailed metrics and benchmarks with such a &quot;big&quot; (loud) update.

It&#x27;s seemed that way for the last year.  The only real improvements have been in the chat apps themselves (internet access, function calling). Until AI gets past the pre-training problem, it&#x27;ll stagnate.

<a href="https:&#x2F;&#x2F;epoch.ai&#x2F;data-insights&#x2F;llm-apis-accuracy-runtime-tradeoff" rel="nofollow">https:&#x2F;&#x2F;epoch.ai&#x2F;data-insights&#x2F;llm-apis-accuracy-runtime-tra...</a>It is easier to get from 0% accurate to 99% accurate, than it is to get from 99% accurate to 99.9% accurate.This is like the classic 9s problem in SRE. Each nine is exponentially more difficult.How easy do we really think it will be for an LLM to get 100% accurate at physics, when we don&#x27;t even know what 100% right is, and it&#x27;s theoretically possible it&#x27;s not even physically possible?

Is there a graph somewhere that illustrates it?

I don&#x27;t agree, the only thing thing that would shock me about this model is if it didn&#x27;t hallucinate.I think the actual effect of releasing more models every month has been to confuse people that progress is actually happening. Despite claims of exponentially improved performance and the ability to replace PhDs, doctors, and lawyers, it still routinely can&#x27;t be trusted the same as the original ChatGPT, despite years of effort.

i think this is more an effect of releasing a model every other month with gradual improvements. if there was no o-series&#x2F;other thinking models on the market - people would be shocked by this upgrade. the only way to keep up with the market is to release improvements asap

IMO is not breakthrough, if you craft proper prompts you can excel imo with 2.5 Pro. Paper : <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2507.15855" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2507.15855</a>. Google just put whole computational power with very high quality data. It was test-time scaling. Why it didn&#x27;t solve problem 6 as well?Yes, it was breakthrough but saturated quickly. Wait for next breakthrough. If they can build adapting weights in llm we can talk different things but test time scaling coming to end with increasing hallucination rate. No sign for AGI.

Layman&#x27;s perspective: we had hints of reasoning from the initial release of ChatGPT when people figured out you could prompt &quot;think step by step&quot; to drastically increase problem solving performance. Then yeah a year+ later it was cleverly incorporated into model training.

We still don&#x27;t have reasoning. We have synthetic text extrusion machines priming themselves to output text that looks a certain way by first generating some extra text that gets piped back into their own input for a second round.

Before last year we didn&#x27;t have reasoning. It came with QuietSTaR, then we got it in the form of O1 and then it became practical with DeepSeek&#x27;s paper in January.So we&#x27;re only about a year since the last big breakthrough.I think we got a second big breakthrough with Google&#x27;s results on the IMO problems.For this reason I think we&#x27;re very far from hitting a wall. Maybe &#x27;LLM parameter scaling is hitting a wall&#x27;. That might be true.

I don&#x27;t remember as a big fan of DeepSeek.

And they said &quot;it&#x27;s over&quot; millions of times. What they mean is the exponential expectations are done.

Deepseek was never SOTA, it was a big deal because it was from China but it wasn&#x27;t a breakthrough in any sense

Just an absurd statement when DeepSeek had its moment in January.A whole 8 months ago.

Not really, it&#x27;s just that our benchmarks are not good at showing how they&#x27;ve improved. Those that regularly try out LLMs can attest to major improvements in reliability over the past year.

I mean test-time scaling coming to end, there are many open rooms for next thing.

GPT5 doesn&#x27;t add any cues to whether we hit the wall, as OpenAI only needs to go one step beyond the competition. They are market leaders and more profitable than the others, so it&#x27;s possible are not showing us everything they have, until they really need to.

I think the llm wall was hit a while ago and the jumps have been around finessing llms in novel ways for a better result. But the core is still very much the same it has been for a while.The crypto level hype claims are all bs and we all knew that but i do use an llm more than google now which is the there there so to speak.This does feel like a flatlining of hype tho which is great because idk if i could take the ai hype train for much longer.

Yeah, but OpenAI is concerned with getting on a path to making money, as their investors will eventually run out of money to light on fire, so...

Personally, I am more concerned about accuracy than speed.

Significant cost reduction while providing the same performance seems pretty big to me?Not sure why a more efficient&#x2F;scalable model isn&#x27;t exciting

Yeah, this very much feels like &quot;we have made a more efficient&#x2F;scalable model and we&#x27;re selling it as the new shiny but it&#x27;s really just an internal optimization to reduce cost&quot;

The silent victory here is this seems like it is being built to be faster and cheaper than o3 while presenting a reasonable jump, which is an important jump in scaling lawOn the other hand if it&#x27;s just getting bigger and slower it&#x27;s not a good sign for LLMs

So the benchmark graphs they have shown so far in the stream appears to show that GPT-5 is WORSE than other models unless you use thinking?

I am not sure that we are not presented with a Catch-22. Yes, life might likely be better for developers and other careers if AI fails to live up to expectations. However, a lot companies, i.e., many of our employers, have invested a lot of money in these products. In the event AI fails, I think the stretched rubber band of economics will slap back hard. So, many might end up losing their jobs (and more) anyway.

Yes, it&#x27;s bad. Because we&#x27;re all dying of cancer, heart disease and auto-immune disease, not to mention traffic accidents and other random killers that AI could warn us about and fix.I don&#x27;t mind losing my programming job in exchange for being able to go to the pharmacy for my annual anti-cancer pill.

Is it bad I quietly hope AI fails to live up to expectations?

Sounds more like “I’m hoping it doesn’t eat my lunch”, but everyone else be damned.

No, it&#x27;s not bad to hope that your industry and source of income isn&#x27;t about to be gutted by corporations

Seeing the system card <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;pdf&#x2F;8124a3ce-ab78-4f06-96eb-49ea29ffb52f&#x2F;gpt5-system-card-aug7.pdf" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;pdf&#x2F;8124a3ce-ab78-4f06-96eb-49ea29ffb...</a>there is some improvements in some benchs and not else worthy of note in coding. i only took a peek though so i might be wrong

You are losing your job either way. Either AI will successfully take it, or as you no doubt read in the article yesterday, AI is the only thing propping up the economy, so the jobs will also be cut in the fallout if AI fails to deliver.

What&#x27;s bad about not wanting to lose your job?

Is it bad that I hope it&#x27;s not a significant improvement in coding?

Driver license and selfies. Also still not available in API after doing that! Edit: I do have access now via API.

Does that even work? 
it required passport, personal details, what else?

API usage requires organization verification with your ID :(.

They claim it thinks the &quot;perfect amount&quot; but there is no perfect amount. It all depends on willingness to pay, latency tolerance, etc.

The dev blog makes it sound like they’re aiming more for “AI teammate” than just another upgrade. That said, it’s hard to tell how much of this is real improvement vs better packaging. Benchmarks are cherry-picked as usual, and there’s not much comparison to other models. Curious to hear how it performs in actual workflows.

My thought exactly.-Diminished returns.-... here&#x27;s hoping it leads to progress.-

im sure i am repeating someone else but sounds like we&#x27;re coming over the s-curve

Well it was their choice to call it GPT 5 and not GPT 4.2.

If everyone else had stayed silent as well, then I would agree. But as it is right now they are juuust about managing to match the current pace of the other contenders. 
Which actually is fine, but they have previously set quite high expectations. So some will probably be disappointed at this.

If they had stayed silent since GPT-4, nobody would care what OpenAI was releasing as they would have become completely irrelevant compared to Gemini&#x2F;Claude.

I mean that&#x27;s just the consequence of releasing a new model every couple months. If Open AI stayed mostly silent since the GPT-4 release (like they did for most iterations) and only now released 5 then nobody would be complaining about weak gains in benchmarks.

I am hoping there is a &quot;One more thing&quot; that shows the pro version with great benchmark scores

It is at least much cheaper and seems faster.They also announced gpt-5-pro but I haven&#x27;t seen benchmarks on that yet.

This same leaderboard lists a bunch of models, including 4o, beating out Opus 4, which seems off.

&quot;+100 points&quot; sounds like a lot until you do the ELO math and see that means 1 out of 3 people still preferred Claud Opus 4&#x27;s response. Remember 1 out of 2 would place the models dead even.

That eval hasn&#x27;t been relevant for a while now. Performance there just doesn&#x27;t seem to correlate well with real-world performance.

What does +75 arbitrary points mean in practice? Can we come up with units that relate to something in the real world.

GPT-5 is #1 on WebDev Arena with +75 pts over Gemini 2.5 Pro and +100 pts over Claude Opus 4:<a href="https:&#x2F;&#x2F;lmarena.ai&#x2F;leaderboard" rel="nofollow">https:&#x2F;&#x2F;lmarena.ai&#x2F;leaderboard</a>

Then why increment the version number here? This is clearly styled like a &quot;mic drop&quot; release but without the numbers to back it up. It&#x27;s a really bad look when comparing the crazy jump from GPT3 to GPT4 to this slight improvement with GPT5.

He said that because even then he saw the writing on the wall that LLMs will plateau.

&gt; Sam said maybe two years ago that they want to avoid &quot;mic drop&quot; releases, and instead want to stick to incremental steps.He also said that AGI was coming early 2025.People that can&#x27;t stop drinking the kool aid are really becoming ridiculous.

Sam said maybe two years ago that they want to avoid &quot;mic drop&quot; releases, and instead want to stick to incremental steps.This is day one, so there is probably another 10-20% in optimizations that can be squeezed out of it in the coming months.

I think this is pretty much what we&#x27;ve already seen happening, in fact.

I have a suspicion that while the major AI companies have been pretty samey and competing in the same space for a while now, the market is going to force them to differentiate a bit, and we&#x27;re going to see OpenAI begin to lose the race toward extremely high levels of intelligence instead choosing to focus on justifying their valuations by optimizing cost and for conversational&#x2F;normal intelligence&#x2F;personal assistant use-cases. After all, most of their users just want to use it to cheat at school, get relationship advice, and write business emails. They also have Ive&#x27;s company to continue investing in.Meanwhile, Anthropic &amp; Google have more room in their P&#x2F;S ratios to continue to spend effort on logarithmic intelligence gains.Doesn&#x27;t mean we won&#x27;t see more and more intelligent models out of OpenAI, especially in the o-series, but at some point you have to make payroll and reality hits.

Also, the code demos are all using GPT-5 MAX on Cursor. Most of us will not be able to use it like that all the time. They should have showed it without MAX mode as well

I’m more worried about how they still confidently reason through things incorrectly all the time, which isn’t quite the same as hallucination, but it’s in a similar vein.

The hallucination benchmarks did show major improvement. We know existing benchmarks are nearly useless at this point. It&#x27;s reliability that matters more.

Is he suggesting his company is designed with a womp rat sized opening that if you shoot a bullet into makes the whole thing explode?

He also said he had an existential crisis that he was completely useless now at work.

&gt; I know they seemed to be trying to temper our expectations leading up to thisBefore the release of the model Sam Altman tweeted a picture of the Death Star appearing over the horizon of a planet.

Yeah I think that throwing more and more compute at the same training data produces smaller and smaller gains.Maybe quantum compute would be significant enough of a computing leap to meaningfully move the needle again.

I&#x27;m not sure what &quot;10% performance gain&quot; is supposed to mean here; but moving from &quot;It does a decent job 95% of the time but screws it up 5%&quot; to &quot;It does a decent job 98% of the time and screws it up 2%&quot; to &quot;It does a decent job 99.5% of the time and only screws it up 0.5%&quot; are major qualitative improvements.

Law of diminishing returns.We’re talking about less than a 10% performance gain, for a shitload of data, time, and money investment.

Watching the livestream now, the improvement over their current models on the benchmarks is very small. I know they seemed to be trying to temper our expectations leading up to this, but this is much less improvement than I was expecting

I mean, I don’t disagree. Why even bother with a non-thinking mode?

Um... if I want an intelligence, when would I not want it to think?

The benchmarks in the stream appears to show that GPT-5 performs WORSE than other models unless you enable thinking?