It could be, but there’s so much hype surrounding the GPT-5 release that I’m not sure whether their internal models will live up to it.For GPT-5 to dwarf these just-released models in importance, it would have to be a huge step forward, and I’m still doubting about OpenAI’s capabilities and infrastructure to handle demand at the moment.

The catch is that it only has ~5 billion active params so should perform worse than the top Deepseek and Qwen models, which have around 20-30 billion, unless OpenAI pulled off a miracle.

&gt; What’s the catch?Probably GPT5 will be way way better. If alpha&#x2F;beta horizon are early previews of GPT5 family models, then coding should be &gt; opus4 for modern frontend stuff.

It&#x27;s definitely interesting how the comments from right after the models were released were ecstatic about &quot;SOTA performance&quot; and how it is &quot;equivalent to o3&quot; and then comments like yours hours later after having actually tested it keep pointing out how it&#x27;s garbage compared to even the current batch of open models let alone proprietary foundation models.Yet another data point for benchmarks being utterly useless and completely gamed at this stage in the game by all the major AI developers.These companies are clearly are all very aware that the initial wave of hype at release is &quot;sticky&quot; and drives buzz&#x2F;tech news coverage while real world tests take much longer before that impression slowly starts to be undermined by practical usage and comparison to other models. Benchmarks with wildly over confident naming like &quot;Humanity&#x27;s Last Exam&quot; aren&#x27;t exactly helping with objectivity either.

The catch is that performance is not actually comparable to o4-mini, never mind o3.When it comes to LLMs, benchmarks are bullshit. If they sound too good to be true, it&#x27;s because they are. The only thing benchmarks are useful for is preliminary screening - if the model does especially badly in them it&#x27;s probably not good in general. But if it does good in them, that doesn&#x27;t really tell you anything.

Open weight models from OpenAI with performance comparable to that of o3 and o4-mini in benchmarks… well, I certainly wasn’t expecting that.What’s the catch?

Maybe someone got tired of waiting paid them to release something actually open

their promise to release an open weights model predates this round of funding by, iirc, over half a year.

Undercutting other frontier models with your open source one is not an anti-investor move.It is what China has been doing for a year plus now. And the Chinese models are popular and effective, I assume companies are paying for better models.Releasing open models for free doesn’t have to be charity.

Ha. Secure funding and proceed to immediately make a decision that would likely conflict viscerally with investors.

That&#x27;s what I thought too until Qwen-Image was released

Honestly, it&#x27;s a tradeoff. If you can reduce the size and make a higher quality in specific tasks, that&#x27;s better than a generalist that can&#x27;t run on a laptop or can&#x27;t compete at any one task.We will know soon the actual quality as we go.

&gt; so better to take a competitive model and latch on vision&#x2F;audioCan this be done by a third party or would it have to be OpenAI?

The community can always figure out hooking it up to other modalities.Native might be better, but no native multimodal model is very competitive yet, so better to take a competitive model and latch on vision&#x2F;audio

Text only, when local multimodal became table stakes last year.

Listed performance of ~5 points less than o3 on benchmarks is pretty impressive.Wonder if they feel the bar will be raised soon (GPT-5) and feel more comfortable releasing something this strong.

I believe their competition is from chinese companies , for some time now

There&#x27;s nothing new here in terms of architecture. Whatever secret sauce is in the training.

Maverick and Scout were not great, even with post-training in my experience, and then several Chinese models at multiple sizes made them kind of irrelevant (dots, Qwen, MiniMax)If anything this helps Meta: another model to inspect&#x2F;learn from&#x2F;tweak etc. generally helps anyone making models

I&#x27;ll accept Meta&#x27;s frontier AI demise if they&#x27;re in their current position a year from now. People killed Google prematurely too (remember Bard?), because we severely underestimate the catch-up power bought with ungodly piles of cash.

Meta is so cooked, I think most enterprises will opt for OpenAI or Anthropic and others will host OSS models themselves or on AWS&#x2F;infra providers.

<pre><code> OAI open source
</code></pre>
Yeah. This certainly was not on my bingo card.

I also tried it for a full 100K credits (Wasted in 2 hours btw which is silly!).Compared to both Udio and Suno, it&#x27;s very very bad.. both at compositions, matching lyrics to music, keeping tempo and as soon as there&#x27;s any distorted instruments like guitars or live, quality goes to radio-level.

&gt;These all seem made for non-musicians to make a very cookie cutter song from a specific genre.This is my main problem with AI music at the moment, I&#x27;d love it if I had proper creative control as a musician that&#x27;d be amazing but a lot of the time it&#x27;s just straight up slop generation.

wow I just listened to Eleven Music do flamenco singing. That is incredible.Edit. I just tried it though and less impressed now. We are really going to need major music software to get on board before we have actual creative audio tools. These all seem made for non-musicians to make a very cookie cutter song from a specific genre.

Wow, today is a crazy AI release day:- OAI open source- Opus 4.1- Genie 3- ElevenLabs Music

Even before today, the last week or so, it&#x27;s been clear for a couple reasons, that GPT-5&#x27;s release was imminent.

Even without an imminent release it&#x27;s a good strategy. They&#x27;re getting pressure from Qwen and other high performing open-weight models. without a horse in the race they could fall behind in an entire segment.There&#x27;s future opportunity in licensing, tech support, agents, or even simply to dominate and eliminate. Not to mention brand awareness, If you like these you might be more likely to approach their brand for larger models.

How much hype do we anticipate with the release of GPT-5 or whichever name to be included?  And how many new features?

Is this the stealth models horizon alpha and beta? I was generally impressed with them(although I really only used it in chats rather than any code tasks). In terms of chat I increasingly see very little difference between the current SOTA closed models and their open weight counterparts.

&gt; I can&#x27;t think of any reason they&#x27;d release this unless they were about to announce something which totally eclipses itGiven it&#x27;s only around 5 billion active params it shouldn&#x27;t be a competitor to o3 or any of the other SOTA models, given the top Deepseek and Qwen models have around 30 billion active params. Unless OpenAI somehow found a way to make a model with 5 billion active params perform as well as one with 4-8 times more.

Undoubtedly. It would otherwise reduce the perceived value of their current product offering.The question is how much better the new model(s) will need to be on the metrics given here to feel comfortable making these available.Despite the loss of face for lack of open model releases, I do not think that was a big enough problem t undercut commercial offerings.

Thursday<a href="https:&#x2F;&#x2F;manifold.markets&#x2F;Bayesian&#x2F;on-what-day-will-gpt5-be-released" rel="nofollow">https:&#x2F;&#x2F;manifold.markets&#x2F;Bayesian&#x2F;on-what-day-will-gpt5-be-r...</a>

So this confirms a best-in-class model release within the next few days?From a strategic perspective, I can&#x27;t think of any reason they&#x27;d release this unless they were about to announce something which totally eclipses it?

thanks, this part is clear to me.but I need to understand 20 x 1k token throughputI assume it just might be too early to know the answer

You also need space in VRAM for what is required to support the context window; you might be able to do a model that is 14GB in parameters with a small (~8k maybe?) context window on a 16GB card.

gpt-oss:20b is ~14GB on disk [1] so fits nicely within a 16GB VRAM card.[1] <a href="https:&#x2F;&#x2F;ollama.com&#x2F;library&#x2F;gpt-oss">https:&#x2F;&#x2F;ollama.com&#x2F;library&#x2F;gpt-oss</a>

&gt; assuming I need 1k tokens&#x2F;second throughput (on each, so 20 x 1k)3.6B activated at Q8 x 1000 t&#x2F;s = 3.6TB&#x2F;s just for activated model weights (there&#x27;s also context). So pretty much straight to B200 and alike. 1000 t&#x2F;s per user&#x2F;agent is way too fast, make it 300 t&#x2F;s and you could get away with 5090&#x2F;RTX PRO 6000.

You can batch only if you have distinct chat in parallel,

An A100 is probably 2-4k tokens&#x2F;second on a 20B model with batched inference.Multiply the number of A100&#x27;s you need as necessary.Here, you don&#x27;t really need the ram. If you could accept fewer tokens&#x2F;second, you could do it much cheaper with consumer graphics cards.Even with A100, the sweet-spot in batching is not going to give you 1k&#x2F;process&#x2F;second. Of course, you could go up to H100...

Groq is offering 1k tokens per second for the 20B model.You are unlikely to match groq on off the shelf hardware as far as I&#x27;m aware.

How Macs compare to RTXs for this? I.e. what numbers can be expected from Mac mini&#x2F;Mac Studio with 64&#x2F;128&#x2F;256&#x2F;512GB of unified memory?

(answer for 1 inference)
Al depends on the context length you want to support as the activation memory will dominate the requirements. For 4096 tokens you will get away with 24GB (or even 16GB), but if you want to go for the full 131072 tokens you are not going to get there with a 32GB consumer GPU like the 5090. You&#x27;ll need to spring for at the minimum an A6000 (48GB) or preferably an RTX 6000 Pro (96GB).Also keep in mind this model does use 4-bit layers for the MoE parts. Unfortunately native accelerated 4-bit support only started with Blackwell on NVIDIA. So your 3090&#x2F;4090&#x2F;A6000&#x2F;A100&#x27;s are not going to be fast. An RTX 5090 will be your best starting point in the traditional card space. Maybe the unified memory minipc&#x27;s like the Spark systems or the Mac mini could be an alternative, but I do not know them enough.

<a href="https:&#x2F;&#x2F;apxml.com&#x2F;tools&#x2F;vram-calculator" rel="nofollow">https:&#x2F;&#x2F;apxml.com&#x2F;tools&#x2F;vram-calculator</a>

Disclamer: probably dumb questionsso, the 20b model.Can someone explain to me what I would need to do in terms of resources (GPU, I assume) if I want to run 20 concurrent processes, assuming I need 1k tokens&#x2F;second throughput (on each, so 20 x 1k)Also, is this model better&#x2F;comparable for information extraction compared to gpt-4.1-nano, and would it be cheaper to host myself 20b?

Good move for China. No one was going to trust their models outright, now they not only have a track record, but they were able to undercut the value of US models at the same time.

It seems like the various Chinese companies are far outplaying Meta at that game. It remains to be seen if they’re able to throw money at the problem to turn things around.

Meta&#x27;s goal with Llama was to target OpenAI with a &quot;scorched earth&quot; approach by releasing powerful open models to disrupt the competitive landscape. Looks like OpenAI is now using the same playbook.

Is there any details about hardware requirements for a sensible tokens per second for each size of these models?

Eh 20B is pretty managable, 32GB of regular RAM and some VRAM will run you a 30B with partial offloading. After that it gets tricky.

with quantization, 20B fits effortlessly in 24GBwith quantization + CPU offloading, non-thinking models run kind of fine (at about 2-5 tokens per second) even with 8 GB of VRAMsure, it would be great if we could have models in all sizes imaginable (7&#x2F;13&#x2F;24&#x2F;32&#x2F;70&#x2F;100+&#x2F;1000+), but 20B and 120B are great.

I am not at all disappointed. I&#x27;m glad they decided to go for somewhat large but reasonable to run models on everything but phones.Quite excited to give this a try

I wouldn’t stop at 16GB right now.24 is the lowest I would go. Buy a used 3090. Picked one up for $700 a few months back, but I think they were on the rise then.The 3000 series can’t do FP8fast, but meh. It’s the OOM that’s tough, not the speed so much.

A small part of me is considering going from a 4070 to a 16GB 5060 Ti just to avoid having to futz with offloadingI&#x27;d go for an ..80 card but I can&#x27;t find any that fit in a mini-ITX case :(

Native FP4 quantization means it requires half as many bytes as parameters, and will have next to zero quality loss (on the order of 0.1%) compared to using twice the VRAM and exponentially more expensive hardware. FP3 and below gets messier.

I&#x27;m disappointed that the smallest model size is 21B parameters, which strongly restricts how it can be run on personal hardware. Most competitors have released a 3B&#x2F;7B model for that purpose.For self-hosting, it&#x27;s smart that they targeted a 16GB VRAM config for it since that&#x27;s the size of the most cost-effective server GPUs, but I suspect &quot;native MXFP4 quantization&quot; has quality caveats.

Most people don&#x27;t really care all that much about the distinction. It comes across to them as linguistic pedantry and they downvote it to show they don&#x27;t want to hear&#x2F;read it.

I don’t know why you got so much downvoted, these models are not open-source&#x2F;open-recipes. They are censored open weights models. Better than nothing, but far from being Open

That’s why they are called open weight and not open source.

Correct. I agree with them, most of the open weight models are not open source.

by your definition most of the current open weight  models would not qualify

Keep fighting the &quot;open weights&quot; terminology fight, because diluting the term open-source for a blob of neural network weights (even inference code is open-source) is not open-source.

The system prompt is an inference parameter, no?

To many people there&#x27;s an important distinction between &quot;open source&quot; and &quot;open weights&quot;. I agree with the distinction, open source has a particular meaning which is not really here and misuse is worth calling out in order to prevent erosion of the terminology.Historically this would be like calling a free but closed-source application &quot;open source&quot; simply because the application is free.

The parent’s point is that open weight is not the same as open source.Rough analogy:SaaS = AI as a serviceLocally executable closed-source software = open-weight modelOpen-source software = open-source model (whatever allows to reproduce the model from training data)

I don&#x27;t have the $XXbn to train a model, but I certainly would like to know what the training data consists of.

Do you need to see the source code used to compile this binary before you can know it is open? Do you have enough disk storage and RAM available to compile Chromium on your laptop? I don&#x27;t.

I agree with OP - the weights are more akin to the binary output from a compiler. You can&#x27;t see how it works, how it was made, you can&#x27;t freely manipulate with it, improve it, extend it etc. It&#x27;s like having a binary of a program. The source code for the model was the training data. The compiler is the tooling that can train a module based on a given set of training data. For me it is not critical for an open source model that it is ONLY distributed in source code form. It is fine that you can also download just the weights. But it should be possible to reproduce the weights - either there should be a tar.gz ball with all the training data, or there needs to be a description&#x2F;scripts of how one could obtain the training data. It must be reproducible for someone willing to invest the time, compute into it even if 99.999% use only the binary. This is completely analogous to what is normally understood by open source.

Is your point really that- &quot;I need to see all data downloaded to make this model, before I can know it is open&quot;? Do you have $XXB worth of GPU time to ingest that data with a state of the art framework to make a model? I don&#x27;t. Even if I did, I&#x27;m not sure FB or Google are in any better position to claim this model is or isn&#x27;t open beyond the fact that the weights are there.They&#x27;re giving you a free model. You can evaluate it. You can sue them. But the weights are there. If you dislike the way they license the weights, because the license isn&#x27;t open enough, then sure, speak up, but because you can&#x27;t see all the training data??! Wtf.

What is the source that&#x27;s open? Aren&#x27;t the models themselves more akin to compiled code than to source code?

No, it&#x27;s open weight. You wouldn&#x27;t call applications with only Apache 2.0-licensed binaries &quot;open source&quot;. The weights are not the &quot;source code&quot; of the model, they are the &quot;compiled&quot; binary, therefore they are not open source.However, for the sake of argument let&#x27;s say this release should be called open source.Then what do you call a model that also comes with its training material and tools to reproduce the model? Is it also called open source, and there is no material difference between those two releases? Or perhaps those two different terms should be used for those two different kind of releases?If you say that actually open source releases are impossible now (for mostly copyright reasons I imagine), it doesn&#x27;t mean that they will be perpetually so. For that glorious future, we can leave them space in the terminology by using the term open weight. It is also the term that should not be misleading to anyone.

It’s open source, but it’s a binary-only release.It’s like getting a compiled software with an Apache license. Technically open source, but you can’t modify and recompile since you don’t have the source to recompile. You can still tinker with the binary tho.

&gt; It&#x27;s apache2.0, so by definition it&#x27;s open source.That&#x27;s not true by any of the open source definitions in common use.Source code (and, optionally, derived binaries) under the Apache 2.0 license are open source.But compiled binaries (without access to source) under the Apache 2.0 license are not open source, even though the license does give you some rights over what you can do with the binaries.Normally the question doesn&#x27;t come up, because it&#x27;s so unusual, strange and contradictory to ship closed-source binaries with an open source license. Descriptions of which licenses qualify as open source licenses assume the context that of course you have the source or could get it, and it&#x27;s a question of what you&#x27;re allowed to do with it.The distinction is more obvious if you ask the same question about other open source licenses such as GPL or MPL. A compiled binary (without access to source) shipped with a GPL license is not by any stretch open source. Not only is it not in the &quot;preferred form for editing&quot; as the license requires, it&#x27;s not even permitted for someone who receives the file to give it to someone else and comply with the license. If someone who receives the file can&#x27;t give it to anyone else (legally), then it&#x27;s obvioiusly not open source.

It&#x27;s apache2.0, so by definition it&#x27;s open source. Stop pushing for training data, it&#x27;ll never happen, and there&#x27;s literally 0 reason for it to happen (both theoretical and practical). Apache2.0 IS opensource.

Please don&#x27;t use the open-source term unless you ship the TBs of data downloaded from Anna&#x27;s Archive that are required do build it yourself. And dont forget all the system prompts to censor the multiple topics that they don&#x27;t want you to see.

Agreed, these models seem relatively mediocre to Qwen3 &#x2F; GLM 4.5

This might mean there&#x27;s no moat for anything.Kind of a P=NP, but for software deliverability.

It only seems like that if you haven&#x27;t been following other open source efforts. Models like Qwen perform ridiculously well and do so on very restricted hardware. I&#x27;m looking forward to seeing benchmarks to see how these new open source models compare.

in that era, OpenAI and Anthropic were still deluding themselves into thinking they would be the &quot;stewards&quot; of generative AI, and the last US administration was very keen on regoolating everything under the sun, so &quot;safety&quot; was just an angle for regulatory capture.God bless China.

I feel like most of the safety concerns ended up being proven correct, but there&#x27;s so much money in it that they decided to push on anyway full steam ahead.AI did get used for fake news, propaganda, mass surveillance, erosion of trust and sense of truth, and mass spamming social media.

Right? I still remember the safety outrage of releasing Llama. Now? My 96 GB of (V)RAM MacBook will be running a 120B parameter frontier lab model. So excited to get my hands on the MLX quants and see how it feels compared to GLM-4.5-air.

here&#x27;s a quick recording from the 20b model on my 128GB M4 Max MBP: <a href="https:&#x2F;&#x2F;asciinema.org&#x2F;a&#x2F;AiLDq7qPvgdAR1JuQhvZScMNr" rel="nofollow">https:&#x2F;&#x2F;asciinema.org&#x2F;a&#x2F;AiLDq7qPvgdAR1JuQhvZScMNr</a>and the 120b: <a href="https:&#x2F;&#x2F;asciinema.org&#x2F;a&#x2F;B0q8tBl7IcgUorZsphQbbZsMM" rel="nofollow">https:&#x2F;&#x2F;asciinema.org&#x2F;a&#x2F;B0q8tBl7IcgUorZsphQbbZsMM</a>I am, um, floored

Here&#x27;s a 4bit 70B parameter model, <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=5ktS0aG3SMc" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=5ktS0aG3SMc</a> (deepseek-r1:70b Q4_K_M) on a M4 Max 128 GB. Usable, but not very performant.

GLM-4.5-air produces tokens far faster than I can read on my MacBook. That&#x27;s plenty fast enough for me, but YMMV.

On a M1 MacBook Air with 8GB, I got this running Gemma 3n:12.63 tok&#x2F;sec • 860 tokens • 1.52s to first tokenI&#x27;m amazed it works at all with such limited RAM

When people talk about running a (quantized) medium-sized model on a Mac Mini, what types of latency and throughput times are they talking about? Do they mean like 5 tokens per second or at an actually usable speed?

aider uses Playwright. I don&#x27;t know what everybody is using but that&#x27;s a good starting point.

Using LM-Studio as the frontend and Playwright-powered MCP tools for browser access. I&#x27;ve had success with one such MCP: <a href="https:&#x2F;&#x2F;github.com&#x2F;instavm&#x2F;coderunner" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;instavm&#x2F;coderunner</a> It has a tool called navigate_and_get_all_visible_text, for example.Here&#x27;s a demo of this functionality: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=9mLrGcuDifo" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=9mLrGcuDifo</a>

What&#x27;s the easiest way to get these local models browsing the web right now?

It&#x27;s only got around 5 billion active parameters; it&#x27;d be a miracle if it was competitive at coding with SOTA models that have significantly more.

That&#x27;s SVGBench, which is a useful benchmark but isn&#x27;t much of a test of general coding

Qwen3 Coder is 4x its size! Grok 3 is over 22x its size!What does the resource usage look like for GLM 4.5 Air? Is that benchmark in FP16? GPT-OSS-120B will be using between 1&#x2F;4 and 1&#x2F;2 the VRAM that GLM-4.5 Air does, right?It seems like a good showing to me, even though Qwen3 Coder and GLM 4.5 Air might be preferable for some use cases.

Okay I will be honest, I was so hyped up about This model but then I went to localllama and saw it that the:120 B model is worse at coding compared to qwen 3 coder and glm45 air and even grok 3... (<a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1mig58x&#x2F;gptoss120b_below_glm45air_and_qwen_3_coder_at&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1mig58x&#x2F;gptoss1...</a>)

We be running them in PIs  off spare juice in no time, and they be billions given how chips and embedded spreads…

Running a model comparable to o3 on a 24GB Mac Mini is absolutely wild. Seems like yesterday the idea of running frontier (at the time) models locally or on a mobile device was 5+ years out. At this rate, we&#x27;ll be running such models in the next phone cycle.

Unfortunately not, this model is noticeably worse. I imagine horizon is either gpt 5 nano&#x2F;mini.

So 120B was Horizon Alpha and 20B was Horizon Beta?

Wow - I will give it a try then. I&#x27;m cynical about OpenAI minmaxing benchmarks, but still trying to be optimistic as this in 8bit is such a nice fit for apple silicon

GLM-4.5 seems to outperform it on TauBench, too. And it&#x27;s suspicious OAI is not sharing numbers for quite a few useful benchmarks (nothing related to coding, for example).One positive thing I see is the number of parameters and size --- it will provide more economical inference than current open source SOTA.

Was the Qwen model using tools for Humanity&#x27;s Last Exam?

GPQA Diamond: gpt-oss-120b: 80.1%, Qwen3-235B-A22B-Thinking-2507: 81.1%Humanity’s Last Exam: gpt-oss-120b (tools): 19.0%, gpt-oss-120b (no tools): 14.9%, Qwen3-235B-A22B-Thinking-2507: 18.2%

Shameless plug: if someone wants to try it in a nice ui, you could give Msty[1] a try. It&#x27;s private and local.[1]: <a href="https:&#x2F;&#x2F;msty.ai" rel="nofollow">https:&#x2F;&#x2F;msty.ai</a>

Getting lots of 502s from `<a href="https:&#x2F;&#x2F;api.gpt-oss.com&#x2F;chatkit" rel="nofollow">https:&#x2F;&#x2F;api.gpt-oss.com&#x2F;chatkit</a>` at the moment.

Why isn&#x27;t GPT-OSS also offered on the free tier of ChatGPT?

(I helped build the microsite)Our backend is falling over from the load, spinning up more resources!

Does anyone get the demos at <a href="https:&#x2F;&#x2F;www.gpt-oss.com" rel="nofollow">https:&#x2F;&#x2F;www.gpt-oss.com</a> to work, or are the servers down immediately after launch? I&#x27;m only getting the spinner after prompting.

True, but there&#x27;s still a meaningful difference in friction and scale. With closed APIs, OpenAI can monitor for misuse, throttle abuse and deploy countermeasures in real-time. With open weights, a single prompt jailbreak or exploit spreads instantly. No need for ML expertise, just a Reddit post.The risk isn’t that bad actors suddenly become smarter. It’s that anyone can now run unmoderated inference and OpenAI loses all visibility into how the model’s being used or misused. I think that’s the control they’re grappling with under the label of safety.

I think it&#x27;s pointless: if you SFT even their closed source models on a specific enough task, the guardrails disappear.AI &quot;safety&quot; is about making it so that a journalist can&#x27;t get out a recipe for Tabun just by asking.

The repeated safety testing delays might not be purely about technical risks like misuse or jailbreaks. Releasing open weights means relinquishing the control OpenAI has had since GPT-3. No rate limits, no enforceable RLHF guardrails, no audit trail. Unlike API access, open models can&#x27;t be monitored or revoked. So safety may partly reflect OpenAI&#x27;s internal reckoning with that irreversible shift in power, not just model alignment per se. What do you guys think?

The model files contain an exact description of the architecture of the network, there isn&#x27;t anything novel.Given these new models are closer to the SOTA than they are to competing open models, this suggests that the &#x27;secret sauce&#x27; at OpenAI is primarily about training rather than model architecture.Hence why they won&#x27;t talk about the training.

This is super helpful and I had not seen it, thanks so much for sharing! And I hear you on training being an alpha, at the size of the model I wonder how much of this is distillation and using o3&#x2F;o4 data.

Their model card [0] has some information. It is quite a standard architecture though; it&#x27;s always been that their alpha is in their internal training stack.[0] <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;pdf&#x2F;419b6906-9da6-406c-a19d-1bb078ac7637&#x2F;oai_gpt-oss_model_card.pdf" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;pdf&#x2F;419b6906-9da6-406c-a19d-1bb078ac7...</a>

Exciting as this is to toy around with...Perhaps I missed it somewhere, but I find it frustrating that, unlike most other open weight models and despite this being an open release, OpenAI has chosen to provide pretty minimal transparency regarding model architecture and training. It&#x27;s become the norm for LLama, Deepseek, Qwenn, Mistral and others to provide a pretty detailed write up on the model which allows researchers to advance and compare notes.

Not gonna lie but I got sorta goosebumpsI am not kidding but such progress from a technological point of view is just fascinating!

I&#x27;m not sure that&#x27;s a particularly good question for concluding something positive about the &quot;thought for 0.7 seconds&quot; - it&#x27;s such a simple answer, ChatGPT 4o (with no thinking time) immediately answered correctly. The only surprising thing in your test is that o3 wasted 13 seconds thinking about it.

Interesting choice of prompt. None of the local models I have in ollama (consumer mid range gpu) were able to get it right.

When I pay attention to o3 CoT, I notice it spends a few passes thinking about my system prompt. Hard to imagine this question is hard enough to spend 13 seconds on.

How many people are discussing this after one person did 1 prompt with 1 data point for each model and wrote a comment?What is being measured here? For end-to-end time, one model is:t_total = t_network + t_queue + t_batch_wait + t_inference + t_service_overhead

Wow this was actually blazing fast. I prompted &quot;how can the 45th and 47th presidents of america share the same parents?&quot;On ChatGPT.com o3 thought for for 13 seconds, on OpenRouter GPT OSS 120B thought for 0.7 seconds - and they both had the correct answer.

Not even that, even if o3 being marginally better is important for your task (let&#x27;s say) why would anyone use o4-mini? It seems almost 10x the price and same performance (maybe even less): <a href="https:&#x2F;&#x2F;openrouter.ai&#x2F;openai&#x2F;o4-mini" rel="nofollow">https:&#x2F;&#x2F;openrouter.ai&#x2F;openai&#x2F;o4-mini</a>

Non-rhetorically, why would someone pay for o3 api now that I can get this open model from openai served for cheaper? Interesting dynamic... will they drop o3 pricing next week (which is 10-20x the cost[1])?[1] currently $3M in&#x2F; $8M out <a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;pricing" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;pricing</a>

Are the prices staying aligned to the fundamentals (hardware, energy), or is this a VC-funded land grab pushing prices to the bottom?

It&#x27;s funny because I was thinking the opposite, the pricing seems way too high for a 5B parameter activation model.

Wow, that&#x27;s significantly cheaper than o4-mini which seems to be on part with gpt-oss-120b. ($1.10&#x2F;M input tokens, $4.40&#x2F;M output tokens) Almost 10x the price.LLMs are getting cheaper much faster than I anticipated. I&#x27;m curious if it&#x27;s still the hype cycle and Groq&#x2F;Fireworks&#x2F;Cerebras are taking a loss here, or whether things are actually getting cheaper. At this we&#x27;ll be able to run Qwen3-32B level models in phones&#x2F;embedded soon.

Makes sense to me. Inference on these models will be a race to the bottom. Hosting inference themselves will be a waste of compute &#x2F; dollar for them.

It is interesting that openai isn&#x27;t offering any inference for these models.

Cleanshot link for those who don&#x27;t want to go to X: <a href="https:&#x2F;&#x2F;share.cleanshot.com&#x2F;bkHqvXvT" rel="nofollow">https:&#x2F;&#x2F;share.cleanshot.com&#x2F;bkHqvXvT</a>

A few days ago I posted a slowed-down version of the video demo on someone&#x27;s repo because it was unreadably fast due to being sped up.<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44738004">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44738004</a>... today, this is a real-time video of the OSS thinking models by OpenAI on Groq and I&#x27;d have to slow it down to be able to read it. Wild.

I apologize for linking to Twitter, but I can&#x27;t post a video here, so:<a href="https:&#x2F;&#x2F;x.com&#x2F;tekacs&#x2F;status&#x2F;1952788922666205615" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;tekacs&#x2F;status&#x2F;1952788922666205615</a>Asking it about a marginally more complex tech topic and getting an excellent answer in ~4 seconds, reasoning for 1.1 seconds...I am _very_ curious to see what GPT-5 turns out to be, because unless they&#x27;re running on custom silicon &#x2F; accelerators, even if it&#x27;s very smart, it seems hard to justify not using these open models on Groq&#x2F;Cerebras for a _huge_ fraction of use-cases.

I tried this (gpt-oss-120b with Cerebras) with Roo Code. It repeatedly failed to use the tools correctly, and then I got 429 too many requests. So much for the &quot;as fast as I can think&quot; idea!I&#x27;ll have to try again later but it was a bit underwhelming.The latency also seemed pretty high, not sure why. I think with the latency the throughout ends up not making much difference.Btw Groq has the 20b model at 4000 TPS but I haven&#x27;t tried that one.

I really want to try coding with this at 2600 tokens&#x2F;s (from Cerebras). Imagine generating thousands of lines of code as fast as you can prompt. If it doesn&#x27;t work who cares, generate another thousand and try again! And at $.69&#x2F;M tokens it would only cost $6.50 an hour.

Looks like Groq (at 1k+ tokens&#x2F;second) and Fireworks are already live on openrouter: <a href="https:&#x2F;&#x2F;openrouter.ai&#x2F;openai&#x2F;gpt-oss-120b" rel="nofollow">https:&#x2F;&#x2F;openrouter.ai&#x2F;openai&#x2F;gpt-oss-120b</a>$0.15M in &#x2F; $0.6-0.75M outedit: Now Cerebras too at 3,815 tps for $0.25M &#x2F; $0.69M out.

Can&#x27;t wait to see third party benchmarks. The ones in the blog post are quite sparse and it doesn&#x27;t seem possible to fully compare to other open models yet. But the few numbers available seem to suggest that this release will make all other non-multimodal open models obsolete.

I dont see the unsloth files yet but they&#x27;ll be here: <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;unsloth&#x2F;gpt-oss-20b-GGUF" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;unsloth&#x2F;gpt-oss-20b-GGUF</a>Super excited to test these out.The benchmarks from 20B are blowing away major &gt;500b models. Insane.On my hardware.43 tokens&#x2F;sec.I got an error with flash attention turning on. Cant run it with flash attention?31,000 context is max it will allow or model wont load.no kv or v quantization.

Seems to be breaking every agentic tool I&#x27;ve tried so far.Im guessing it&#x27;s going to very rapidly be patched into the various tools.

What a day! Models aside, the Harmony Response Format[1] also seems pretty interesting and I wonder how much of an impact it might have in performance of these models.[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;harmony" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;harmony</a>

Optimistically, there&#x27;s always more crap to get done.

Not to open that can of worms, but in most definitions self-improvement is not an AGI requirement. That&#x27;s already ASI territory (Super Intelligence). That&#x27;s the proverbial skynet (pessimists) or singularity (optimists).

I have always thought that if we can somehow get an AI which is insanely good at coding, so much so that It can improve itself, then through continuous improvements, they will get better models of everything else idkMaybe you guys call it AGI, so anytime I see progress in coding, I think it goes just a tiny bit towards the right directionPlus it also helps me as a coder to actually do some stuff just for the fun. Maybe coding is the only truly viable use of AI and all others are negligible increases.There is so much polarization in the use of AI on coding but I just want to say this, it would be pretty ironic that an industry which automates others job is this time the first to get their job automated.But I don&#x27;t see that as an happening, far from it. But still each day something new, something better happens back to back. So yeah.

Anyone know how long does the context last for running model locally vs running via OpenAPI or Cursor?
My understanding is the model that run on the cloud have much greater context window that what we can have running locally.

The coding seems to be one of the strongest use cases for LLMs. Though currently they are eating too many tokens to be profitable. So perhaps these local models could offload some tasks to local computers.E.g. Hybrid architecture. Local model gathers more data, runs tests, does simple fixes, but frequently asks the stronger model to do the real job.Local model gathers data using tools and sends more data to the stronger model.It

Is this the same model (Horizon Beta) on openrouter or not?
Because I still see Horizon beta available with its codename on openrouter

Test it with a web UI: <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;abidlabs&#x2F;openai-gpt-oss-120b-test" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;abidlabs&#x2F;openai-gpt-oss-120b-t...</a>

Name recognition? Advertisement? Federal grant to beat Chinese competition?There could be many legitimate reasons, but yeah I&#x27;m very surprised by this too. Some companies take it a bit too seriously and go above and beyond too. At this point unless you need the absolute SOTA models because you&#x27;re throwing LLM at an extremely hard problem, there is very little utility using larger providers. In OpenRouter, or by renting your own GPU you can run on-par models for much cheaper.

LLMs are terrible, purely speaking from the business economic side of things.Frontier &#x2F; SOTA models are barely profitable. Previous gen model lose 90% of their value. Two gens back and they&#x27;re worthless.And given that their product life cycle is something like 6-12 months, you might as well open source them as part of sundowning them.

Partially because using their own GPUs is expensive, so maybe offloading some GPU usage

They don&#x27;t because it would kill their data scrapping buisness&#x27;s competitive advantage.

I understand the rationale behind open sourcing llama.cpp. Because it has a lock-in effect.But I don&#x27;t see how open sourcing weights has a lock-in effect. In fact, it seems OpeanAI&#x27;s open models can be run on llama.cpp. So by offereing them, they make llama.cpp even MORE useful. Instead of driving developers towards their own tech.

Zuckerberg explains a few of the reasons here:<a href="https:&#x2F;&#x2F;www.dwarkesh.com&#x2F;p&#x2F;mark-zuckerberg#:~:text=As%20long%20as%20it%27s%20helping%20us%20then%20yeah." rel="nofollow">https:&#x2F;&#x2F;www.dwarkesh.com&#x2F;p&#x2F;mark-zuckerberg#:~:text=As%20long...</a>The short version is that is you give a product to open source, they can and will donate time and money to improving your product, and the ecosystem around it, for free, and you get to reap those benefits. Llama has already basically won that space (the standard way of running open models is llama.cpp), so OpenAI have finally realized they&#x27;re playing catch-up (and last quarter&#x27;s SOTA isn&#x27;t worth much revenue to them when there&#x27;s a new SOTA, so they may as well give it away while it can still crack into the market)

At least in OpenAI&#x27;s case, it raises the bar for potential competition while also implying that what they have behind the scenes is far better.

Oh, wow. Following this line of thinking, what seems like a noble gesture on the surface is a scorched earth approach in reality.

I believe it&#x27;s to create barriers to entry and make the space harder to compete in.There&#x27;s still a ton of value in the lower end of the market by capability, and it&#x27;s easier for more companies to compete in. If you make the cost floor for that basically free you eliminate everyone else&#x27;s ability to make any profit there and then leverage that into building a product that can also compete at the higher end. This makes it harder for a new market entrant to compete by increasing the minimum capability and capital investment required to make a profit in this space.

Why do companies release open source LLMs?I would understand it, if there was some technology lock-in. But with LLMs, there is no such thing. One can switch out LLMs without any friction.

And it&#x27;s already on ollama, it appears: <a href="https:&#x2F;&#x2F;ollama.com&#x2F;library&#x2F;gpt-oss">https:&#x2F;&#x2F;ollama.com&#x2F;library&#x2F;gpt-oss</a>

lm studio immediately released the new appimage with support.

Holy smokes, there&#x27;s already llama.cpp support:<a href="https:&#x2F;&#x2F;github.com&#x2F;ggml-org&#x2F;llama.cpp&#x2F;pull&#x2F;15091" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ggml-org&#x2F;llama.cpp&#x2F;pull&#x2F;15091</a>

You can technically run it on a 8086 assuming you can get access to a big enough storage.More reasonably, you should be able to run the 20B at non-stupidly-slow speed with a 64bit CPU, 8GB RAM, 20GB SSD.

i wish these models had a minimum ram , cpu and gpu size listed on the site instead of high end and medium end pc.

my very early first impression of the 20b model on ollama is that it is quite good, at least for the code I am working on; arguably good enough to drop a subscription or two

It works fine, although with a bit more latency than non-local models. However, swap usage goes way beyond what I’m comfortable with, so I’ll continue to use smaller models for the foreseeable future.Hopefully other quantizations of these OpenAI models will be available soon.

Did you run it the best way possible? im no expert, but I understand it can affect inference time greatly (which format&#x2F;engine is used)

Update: I tried it out. It took about 8 seconds per token, and didn&#x27;t seem to be using much of my GPU (MPU), but was using a lot of RAM. Not a model that I could use practically on my machine.

M2 with 16GB: It&#x27;s slow for me. ~13GB RAM usage, not locking up my mac, but took a very long time thinking and slowly outputting tokens.. I&#x27;d not consider this usable for everyday usage.

Anyone tried running on a Mac M1 with 16GB RAM yet? I&#x27;ve never run higher than an 8GB model, but apparently this one is specifically designed to work well with 16 GB of RAM.

Do you know when this was introduced (or which paper)? AFAIK it&#x27;s not that way in the original transformer paper, or BERT&#x2F;GPT-2

No they’re usually done at each attention layer.

I looked through their torch implementation and noticed that they are applying RoPE to both query and key matrices in every layer of the transformer - is this standard? I thought positional encodings were usually just added once at the first layer

Shoutout to the hn consensus regarding an OpenAI open model release from 4 days ago: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44758511">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=44758511</a>

For some reason I&#x27;m less excited about this that I was with the Qwen models.

From experience, it&#x27;s much more engineering work on the integrator&#x27;s side than on OpenAI&#x27;s. Basically they provide you their new model in advance, but they don&#x27;t know the specifics of your system, so it&#x27;s normal that you do most of the work. 
Thus I&#x27;m particularly impressed by Cerebras: they only have a few models supported for their extreme perf inference, it must have been huge bespoke work to integrate.

To be fair, it&#x27;s with the help of OpenAI. 
They did it together, before the official release.<a href="https:&#x2F;&#x2F;ollama.com&#x2F;blog&#x2F;gpt-oss">https:&#x2F;&#x2F;ollama.com&#x2F;blog&#x2F;gpt-oss</a>

I remember reading Ollama is going closed source now?<a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1meeyee&#x2F;ollamas_new_gui_is_closed_source&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1meeyee&#x2F;ollamas...</a>

It&#x27;s just as easy with LM Studio.All the real heavy lifting is done by llama.cpp, and for the distribution, by HuggingFace.

Orthogonal, but I just wanted to say how awesome Ollama is. It took 2 seconds to find the model and a minute to download and now I&#x27;m using it.Kudos to that team.

I&#x27;m running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok&#x2F;s for 20B model and 12 tok&#x2F;s for 120B model. I&#x27;d say it&#x27;s pretty usable.

I read that as it runs in data centers (H100 GPUs) or high-end desktops&#x2F;laptops (Strix Halo?).

I actually tried to ask the Model about that, then I asked ChatGPT, both times, they just said that it was marketing speak.I was like no. It is false advertising.

Yes, they are late to the party. Maybe they do not want to eat into the RTX Pro 6000 sales. In the meantime, there is the AMD Ryzen™ Al Max+ 395.

Well if nVidia wasn&#x27;t late, it would be runnable on nVidia project Digits.

I love how they frame High-end desktops and laptops as having &quot;a single H100 GPU&quot;.

It may be useless for many use cases given that its policy prevents it for example from providing &quot;advice or instructions about how to buy something.&quot;(I included details about its refusal to answer even after using tools for web searching but hopefully shorter comment means fewer downvotes.)

Can these do image inputs as well? I can&#x27;t find anything about that on the linked page, so I guess not..?

Very sparse benchmarking results released so far. I&#x27;d bet the Chinese open source models beat them on quite a few of them.

Kimi K2 paper said that the model sparsity scales up with parameters pretty well (MoE sparsity scaling law, as they call, basically calling Llama 4 MoE &quot;done wrong&quot;). Hence K2 has 128:1 sparsity.

It&#x27;s convenient to be able to attribute success to things only OpenAI could&#x27;ve done with the combo of their early start and VC money – licensing content, hiring subject matter experts, etc. Essentially the &quot;soft&quot; stuff that a mature organization can do.

Or, you can say, OpenAI has some real technical advancements on stuff besides attn architecture. GQA8, alternating SWA 128 &#x2F; full attn do all seem conventional. Basically they are showing us that &quot;no secret sauce in model arch you guys just sucks at mid&#x2F;post-training&quot;, or they want us to believe this.The model is pretty sparse tho, 32:1.

The native FP4 is one of the most interesting architectural aspects here IMO, as going below FP8 is known to come with accuracy tradeoffs. I&#x27;m curious how they navigated this and how the FP8 weights (if they exist) were to perform.

&gt;They did something to quantize &gt;90% of the model parameters to the MXFP4 format (4.25 bits&#x2F;parameter) to let the 120B model to fit on a single 80GB GPU, which is pretty coolThey said it was native FP4, suggesting that they actually trained it like that; it&#x27;s not post-training quantisation.

Also: attention sinks (although implemented as extra trained logits used in attention softmax rather than attending to e.g. a prepended special token).

You can get similar insights looking at the github repo <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-oss" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;gpt-oss</a>

I think their MXFP4 release is a bit of a gift since they obviously used and tuned this extensively as a result of cost-optimization at scale - something the open source model providers aren&#x27;t doing too much, and also somewhat of a competitive advantage.Unsloth&#x27;s special quants are amazing but I&#x27;ve found there to be lots of trade offs vs full quantization, particularly when striving for best first-shot attempts - which is by far the bulk of LLM use cases. Running a better (larger, newer) model at lower quantization to fit in memory, or with reduced accuracy&#x2F;detail to speed it up both have value, but in the the pursuit of first-shot accuracy there doesn&#x27;t seem to be many companies running their frontier models on reduced quantization. If openAI is in doing this in production that is interesting.

&gt; research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance.same seems to be true for humans

I would guess the “secret sauce” here is distillation: pretraining on an extremely high quality synthetic dataset from the prompted output of their state of the art models like o3 rather than generic internet text. A number of research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance.This would be much more efficient than relying purely on RL post-training on a small model; with low baseline capabilities the insights would be very sparse and the training very inefficient.

Start with the YT series on neural nets and LLMs from 3blue1brown

There is a great 3blue1brown video, but it’s pretty much impossible by now to cover the entire landscape of research. I bet gpt-oss has some great explanations though ;)

Try Andrej Karpathy&#x27;s YouTube videos. I also really liked the Dive into Deep Learning book at d2l.ai

What I’ve sometimes done when trying to make sense of recent LLM research is give the paper and related documents to ChatGPT, Claude, or Gemini and ask them to explain the specific terms I don’t understand. If I don’t understand their explanations or want to know more, I ask follow-ups. Doing this in voice mode works better for me than text chat does.When I just want a full summary without necessarily understanding all the details, I have an audio overview made on NotebookLM and listen to the podcast while I’m exercising or cleaning. I did that a few days ago with the recent Anthropic paper on persona vectors, and it worked great.

Try Microsoft&#x27;s &quot;Generative AI for Beginners&quot; repo on GitHub. The early chapters in particular give a good grounding of LLM architecture without too many assumptions of background knowledge. The video version of the series is good too.

This is a great book (parts of it are available as blog posts from the author if you want to get a taste of it):<a href="https:&#x2F;&#x2F;www.manning.com&#x2F;books&#x2F;build-a-large-language-model-from-scratch" rel="nofollow">https:&#x2F;&#x2F;www.manning.com&#x2F;books&#x2F;build-a-large-language-model-f...</a>

I don&#x27;t know how to ask this without being direct and dumb: Where do I get a layman&#x27;s introduction to LLMs that could work me up to understanding every term and concept you just discussed? Either specific videos, or if nothing else, a reliable Youtube channel?

Model cards, for the people interested in the guts: <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;pdf&#x2F;419b6906-9da6-406c-a19d-1bb078ac7637&#x2F;oai_gpt-oss_model_card.pdf" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;pdf&#x2F;419b6906-9da6-406c-a19d-1bb078ac7...</a>In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level:- both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how they’ve used an older optimization from GPT3, which is alternating between banded window (sparse, 128 tokens) and fully dense attention patterns. It uses RoPE extended with YaRN (for a 131K context window). So they haven’t been taking advantage of the special-sauce Multi-head Latent Attention from Deepseek, or any of the other similar improvements over GQA.- both models are standard MoE transformers. The 120B model (116.8B total, 5.1B active) uses 128 experts with Top-4 routing. They’re using some kind of Gated SwiGLU activation, which the card talks about as being &quot;unconventional&quot; because of to clamping and whatever residual connections that implies. Again, not using any of Deepseek’s “shared experts” (for general patterns) + “routed experts” (for specialization) architectural improvements, Qwen’s load-balancing strategies, etc.- the most interesting thing IMO is probably their quantization solution. They did something to quantize &gt;90% of the model parameters to the MXFP4 format (4.25 bits&#x2F;parameter) to let the 120B model to fit on a single 80GB GPU, which is pretty cool. But we’ve also got Unsloth with their famous 1.58bit quants :)All this to say, it seems like even though the training they did for their agentic behavior and reasoning is undoubtedly very good, they’re keeping their actual technical advancements “in their pocket”.

<a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-oss&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;introducing-gpt-oss&#x2F;</a>

Open models by OpenAI