Live now,OpenAI Spring Update (<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=DQacCB9tDaw" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=DQacCB9tDaw</a>)<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40343950">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40343950</a>

A new &quot;flagship&quot; model with no improvement of intelligence, very disappointed. Maybe this is a strategy for them to mass collect &quot;live&quot; data before they&#x27;re left behind by Google&#x2F;Twitter live data...

<a href="https:&#x2F;&#x2F;youtu.be&#x2F;DQacCB9tDaw" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;DQacCB9tDaw</a>

I think because usability increases so much (use cases of real-time conversation, 
 and video-based coding, presentation feedback at work etc...) they would expect usage to drastically increase hence paying users would actually still have incentive to pay.

5x the capacity threshold is the only thing I heard them mention on the live stream.Though presumably when they are ready to release new models the Plus users will get them first.

Did they mention this in the gpt4o announcement video? I must have missed this

They mentioned an announcement about a new frontier model coming soon. Presumably this will be exclusive to paid users.

&gt; Plus users will have a message limit that is up to 5x greater than free usersfrom <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;gpt-4o-and-more-tools-to-chatgpt-free&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;gpt-4o-and-more-tools-to-chatgpt-fr...</a>

So what&#x27;s the point of paying for ChatGPT Plus? And who on earth chose to make the app Mac only...

I like the demo for sure more than the &quot;reduced latency&quot; Gemini demo [0].[0] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI</a>

Wow this versioning scheme really messed up this prediction market: <a href="https:&#x2F;&#x2F;kalshi.com&#x2F;markets&#x2F;gpt4p5&#x2F;gpt45-released">https:&#x2F;&#x2F;kalshi.com&#x2F;markets&#x2F;gpt4p5&#x2F;gpt45-released</a>

That im-also-a-good-gpt2-chatbot[1] was in fact the new ChatGPT model as people were assuming few days ago here on HN[2].Edit: may be not, name of that bot was just &quot;gpt2-chatbot&quot;. May be that one was some initial iteration?[1] <a href="https:&#x2F;&#x2F;twitter.com&#x2F;LiamFedus&#x2F;status&#x2F;1790064963966370209&#x2F;photo&#x2F;1" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;LiamFedus&#x2F;status&#x2F;1790064963966370209&#x2F;pho...</a>[2] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40199715">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40199715</a>

They mentioned capabilities will be rolled out over the next few weeks: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;DQacCB9tDaw?t=5018" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;DQacCB9tDaw?t=5018</a>

they stated that they will be announcing something new that is on the next frontier (or close to it IIRC) soon. so there will definitely be an incentive to pay because it will be something better than gpt 4o.

I assume the desktop app with voice and vision is rolling out to plus users first?

It is really cool that they are bringing this to free users. It does make me wonder what justifies ChatGPT plus now though...

I&#x27;ve noticed that the GPT-4 model&#x27;s capabilities seem limited compared to its initial release. Others have also pointed this out. I suspect that making the model free might have required reducing its capabilities to meet cost efficiency goals. I&#x27;ll have to try it out to see for myself.

After using it for an hour, I completely agree. Having the faster token rate + their excel analysis actually work is a game changer.

Agree on hoping for an intelligence update, but I think it was clear from teasers that this was not gonna be GPT-5.I&#x27;m not sure how fair it is to classify the new multimodal capabilities as just a gimmick though. I personally haven&#x27;t integrated GPT-4 into my workflow that much and the latency and the fact I have to type a query out is a big reason why.

As I commented in the other thread, really really disappointed there&#x27;s no intelligence update and more of a focus on &quot;gimmicks&quot;. The desktop app did look really good, especially as the models get smarter. Will be canceling my premium as there&#x27;s no real purpose of it until that new &quot;flag ship&quot; model comes out.

It&#x27;s there now! And still 128k context window

I don&#x27;t see 4o or anything new at <a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;models" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;models</a>Overall I am highly skeptical of newer models as they risk worsening the completion quality to make them cheaper for OpenAI to run.

For posterity, GPT-3.5&#x2F;4&#x27;s tokenizer was 100k. The benefit of a larger tokenizer is more efficient tokenization (and therefore cheaper&#x2F;faster) but with massive diminishing returns: the larger tokenizer makes the model more difficult to train but tends to reduce token usage by 10-15%.

Yep. Non-English text gets a much bigger cost drop and speedup compared to English. Has always been a bummer that GPT-4 is like 5x slower and more expensive in Japanese, etc.

Oh interesting, does that mean languages other than English won&#x27;t be paying such a large penalty in terms of token lengths?With previous tokenizers there was a notable increase in the number of tokens needed to represent non-English sentences: <a href="https:&#x2F;&#x2F;simonwillison.net&#x2F;2023&#x2F;Jun&#x2F;8&#x2F;gpt-tokenizers&#x2F;" rel="nofollow">https:&#x2F;&#x2F;simonwillison.net&#x2F;2023&#x2F;Jun&#x2F;8&#x2F;gpt-tokenizers&#x2F;</a>

Does that imply they retrained the foundation model from scratch? I thought changing the tokenization was something you couldn&#x27;t really retrofit to an existing model. I mean sure they might have initialized the weights from the prior GPT-4 model but it&#x27;d still require a lot of retraining.

Yep. Since it’s multimodal. Pictures, text, audio all go into token space.

Lots of those tokens would have to be pixel patches and sound samples right?

That&#x27;s not the right analogy. The &quot;tok&quot; in &quot;Tiktoken&quot; comes from &quot;token&quot;, not &quot;TikTok&quot;.

How are they able to use such a brand name, Tiktoken? Is it because TikTok is Chinese? Tiktoken, it&#x27;s almost like if Apple released the Facebooken library for something entirely unrelated to Facebook.

Tiktoken added support for GPT-4o: <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;tiktoken&#x2F;commit&#x2F;9d01e5670ff50eb74cdb96406c7f3d9add0ae2f8">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;tiktoken&#x2F;commit&#x2F;9d01e5670ff50eb74c...</a>It has an increased vocab size of 200k.

Unless you&#x27;re talking about that sewer clown&#x27;s balloon!

wondering what apple is cooking up and what they&#x27;ll announce next month.by the way the contraction &quot;it&#x27;s&quot; is used to say &quot;it is&quot; or &quot;it has&quot;, it is never a possessive form.

It seems like a very odd decision. It&#x27;s not like OpenAI can&#x27;t afford to develop versions of the application for each OS in parallel.

Why? Just use the API or normal web access version like you have been since ChatGPT became available at all.

Mac only desktop app. Windows version &quot;later this year&quot;. No Linux.Welp there goes my Plus subscription.

First Impressions in no particular order:<pre><code> Being able to interrupt while GPT is talking
 2x faster&#x2F;cheaper
 not really a much smarter model
 Desktop app that can see screenshots
 Can display emotions with and change the sound of &quot;it&#x27;s&quot; voice</code></pre>

[EDIT] The model has since been added to the docsNot seeing it or any of those documented here:<a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;models&#x2F;overview" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;models&#x2F;overview</a>

&gt; Developers can also now access GPT-4o in the API as a text and vision model. GPT-4o is 2x faster, half the price, and has 5x higher rate limits compared to GPT-4 Turbo. We plan to launch support for GPT-4o&#x27;s new audio and video capabilities to a small group of trusted partners in the API in the coming weeks.

Can&#x27;t find info which of these new features are available via the API

I like the humility in your first statement.

With this capability, how close are y&#x27;all to it being able to listen to my pronunciation of a new language (e.g. Italian) and given specific feedback about how to pronounce it like a local?Seems like these would be similar.

&gt; (I work at OpenAI.)Winner of the &#x27;understatement of the week&#x27; award (and it&#x27;s only Monday).Also top contender in the &#x27;technically correct&#x27; category.

Licensing the emotion-intoned TTS as a standalone API is something I would look forward to seeing. Not sure how feasible that would be if, as a sibling comment suggested, it bypasses the text-rendering step altogether.

&quot;(I work at OpenAI.)&quot;Ah yes, also known as being co-founder :)

Random OpenAI question: While the GPT models have become ever cheaper, the price for the tts models have stayed in the $15&#x2F;1Mio char range. I was hoping this would also become cheaper at some point. There&#x27;re so many apps (e.g. language learning) that quickly become too expensive given these prices. With the GPT-4o voice (which sounds much better than the current TTS or TTS HD endpoint) I thought maybe the prices for TTS would go down. Sadly that hasn&#x27;t happened. Is that something on the OpenAI agenda?

How far are we away from something like a helmet with chat GPT and a video camera installed, I imagine this will be awesome for low vision people. Imagine having a guide tell you how to walk to the grocery store, and help you grocery shop without an assistant. Of course you have tons of liability issues here, but this is very impressive

Is it possible to use this as a TTS model? I noticed on the announcement post that this is a single model as opposed to a text model being piped to a separate TTS model.

This is damn near one of the most impressive things, can only imagine especially with live translation and voice synthesis (eleven labs style) you&#x27;d be capable of to integrate with something like teams (select each persons language and do realtime translation to each persons native language, with their own voice and intonations would NUTS)

Will the new voice mode allow mixing languages in sentences?As a language learner, this would be tremendously useful.

I&#x27;ve always been wondering what GPT models lack that makes them &quot;query-&gt;response&quot; only. I&#x27;ve always tried to get chatbots to lose the initially needed query, with no avail. What would It take to get a GPT model to freely generate tokens in a thought like pattern? I think when I&#x27;m alone without query from another human. Why can&#x27;t they?

In my ChatGPT app or on the website I can select GPT-4o as a model, but my model doesn&#x27;t seem to work like the demo.  The voice mode is the same as before and the images come from DALLE and ChatGPT doesn&#x27;t seem to understand or modify them any better than previously.

I couldn’t quite tell from the announcement, but is there still a separate TTS step, where GPT is generating tones&#x2F;pitches that are to be used, or is it completely end to end where GPT is generating the output sounds directly?

<a href="https:&#x2F;&#x2F;community.openai.com&#x2F;t&#x2F;when-i-log-in-to-chatgpt-i-am-prompted-for-a-login-failure-and-a-message-something-went-wrong-please-make-sure-your-devices-date-and-time-are-set-properly&#x2F;508758" rel="nofollow">https:&#x2F;&#x2F;community.openai.com&#x2F;t&#x2F;when-i-log-in-to-chatgpt-i-am...</a>Sorry to hijack, but how the hell can I solve this? I have the EXACT SAME error on two iOS devices (native app only — web is fine), but not on Android, Mac, or Windows.

May I just say this launch was a bit of a mess?The web page implies you can try it immediately. Initially it wasn&#x27;t available.A few hours later it was in both the web UI and the mobile app - I got a popu[ telling me that GPT-4o was available. However nothing seems to be any different. I&#x27;m not given any option to use video as an input, the app can&#x27;t seem to pick up any new info from my voice.I&#x27;m left a bit confused as to what I can do that I couldn&#x27;t do before. I certainly can&#x27;t seem to recreate much of the stuff from the announcement demos.

Who&#x27;s idea was the singing AIs? What specifically did you want to highlight with that part of the demo?I imagine that there is a lot of usage at the HQ, human + AI karaoke?

I can&#x27;t wait to try it out, it sounds too good to be real.It will be fully available in Eu with the GDPR compliance?

(I work at OpenAI.)It&#x27;s really how it works.

However, this looks like it only works with speech - i.e. you can&#x27;t ask it, &quot;What&#x27;s the tune I&#x27;m humming?&quot; or &quot;Why is my car making this noise?&quot;I could be wrong but I haven&#x27;t seen any non-speech demos.

Anyone who has used elevenlabs for voice generation has found this to be the case.  Voice to voice seems like magic.

I asked it to make a bird noise, instead it told me what a bird sounds like with words. True audio to audio should be able to be any noise, a trombone, traffic, a crashing sea, anything. Maybe there is a better prompt there but it did not seem like it.

&gt;The most impressive part is that the voice uses the right feelings and tonal language during the presentation.Consequences of audio2audio (rather than audio &gt;text text&gt;audio). Being able to manipulate speech nearly as well as it manipulates text is something else. This will be a revelation for language learning amongst other things. And you can interrupt it freely now!

I noticed this as well. They gave zero fs about the fit and finish of these videos because they know this is magic in a bottle.

I mention this down thread, but a symptom of a tech product of sufficient advancement is the nature of its introduction matters less and less.Based on the casual production of these videos, the product must be this good.<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40346002">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40346002</a>

Can you tell the current voice model what feelings and tone it should communicate with? If not it isn&#x27;t even comparable, being able to control how it reads things is absolutely revolutionary, that is what was missing from using these AI models as voice actors.

The voice mode was quite good but the latency and start &#x2F; stop has been encumbering.

Slight off-topic, but I noticed you&#x27;ve updated your llm CLI app to work with the 4o model (plus bunch of other APIs through plugins). Kudos for working extremely fast. I&#x27;m really grateful for your tool; I tried many others, but for some reason none clicked as much as your.Link in case other readers are curious: <a href="https:&#x2F;&#x2F;llm.datasette.io" rel="nofollow">https:&#x2F;&#x2F;llm.datasette.io</a>

That was very impressive, but it doesn&#x27;t surprise me much given how good the voice mode is in the ChatGPT iPhone app is already.The new voice mode sounds better, but the current voice mode did also have inflection that made it feel much more natural than most computer voices I&#x27;ve heard before.

It understands tonal language, you can tell it how you want it to talk, I have never seen a model like that before. If you want it to talk like a computer you can tell it to, they did it during the presentation, that is so much better than the old attempts at solving this.

&gt; voice sounds like an over enthusiastic podcast interviewerI believe it can be toned down using system prompts, which they&#x27;ll expose in future iterations

Right... enthusiastic and generally confused. It&#x27;s uncanny valley level expressions. Still better than drab, monotonous speech though.

Genuine People Personalities™, just like in Hitchikers. Perhaps one of the milder forms of &#x27;We Created The Torment Nexus&#x27;.

agree I don&#x27;t get it.  I just want the right information and explained well.  I don&#x27;t want to be social with a robot.

It sounds VERY California. &quot;Its going great!&quot; &quot;Nice choice&quot; &quot;Whats up with the...&quot; all within 10 seconds.(not that this is the most important thing about the announcement at all. Just an aside)

&gt; &quot;over enthusiastic podcast interviewer&quot;Yeh it&#x27;s cringe. I had to stop listening.Why did they make the woman sound like she&#x27;s permanently on the brink of giggling? It&#x27;s nauseating how overstated her pretentious banter is. Somewhere between condescending nanny and preschool teacher. Like how you might talk to a child who&#x27;s at risk of crying so you dial up the positive reinforcement.

exactly. Hope we can customize the voice soon. I want to talk to ultron... 
or the one from mass effect

Right to who? To me, the voice sounds like an over enthusiastic podcast interviewer. Whats wrong with wanting computers to sound like what people think computers should sound like?

I wonder when it will be able to understand that there is more than one human talking to it. It seems like even in today&#x27;s demo if two people are talking, it can&#x27;t tell them apart.

I was in the audience at the event. The only parts where it seemed to get snagged was hearing the audience reaction as an interruption. Which honestly makes the demo even better. It showed that hey, this is live.Magic.

Really? I think interruption and timing in general still seems like a problem that has yet to be solved. It was the most janky aspect of the demos imo.

Crazy that interruption also seems to work pretty smoothly

Seems about as good as Azure&#x27;s Speech Service. I wonder if that&#x27;s what they are using behind the scenes

Somehow it also sounds almost like Dot Matrix from Spaceballs.

Yeah, the female voice especially is really impressive in the demos.  The voice always sounds natural.  The male voice I heard wasn&#x27;t as good.  It wasn&#x27;t terrible, but it had a somewhat robotic feel to it.

This gives me a lot of anxiety but my only recourse is to stop paying attention to AI dev.  Its not going to stop, downside be damned.  The &quot;We&#x27;re working super hard to make these things safe&quot; routine from tech companies, who have largely been content to make messes and then not be held accountable in any significant way, rings pretty hollow for me.  I don&#x27;t want to be a doomer but I&#x27;m not convinced that the upside is good enough to protect us from the downside.

&quot;Right&quot; feelings and tonal language? &quot;Right&quot; for what? For whom?We&#x27;ve already seen how much damage dishonest actors can do by manipulating our text communications with words they don&#x27;t mean, plans they don&#x27;t intend to follow through on, and feelings they don&#x27;t experience. The social media disinfo age has been bad enough.Are you sure you want a machine which is able to manipulate our emotions on an even more granular and targetted level?LLMs are still machines, designed and deployed by humans to perform a task. What will we miss if we anthropomorphize the product itself?

I’m not sure how revolutionary the style is. It can already mimic many styles of writing. It seems like mimicking a cheerful happy assistant, with associated filler words, etc. is very much in-line with what LLM’s are good at.

That&#x27;s the part that really struck me. I thought it was particularly impressive with the Sal Khan maths tutor demo and the one with BeMyEyes. The comment at the end about the dog was an interesting ad-lib.The only slightly annoying thing at the moment is they seem hard to interrupt, which is an important mechanism in conversations. But that seems like a solvable problem. They kind of need to be able to interpret body language a bit to spot when the speaker is about to interrupt.

The most impressive part is that the voice uses the right feelings and tonal language during the presentation. I&#x27;m not sure how much of that was that they had tested this over and over, but it is really hard to get that right so if they didn&#x27;t fake it in some way I&#x27;d say that is revolutionary.

What strong incumbents are there in conversational voice models? Siri? Google Assistant? This is in a completely different league. I can see from the reaction here that people don&#x27;t understand. But they will when they try it.Did you see it translate Italian? Have you ever tried the Google Translate&#x2F;Assistant features for real time translation? They didn&#x27;t train it to be a translator. They didn&#x27;t make a translation feature. They just asked it. It&#x27;s instantly better than every translation feature Google ever released.

What Openai has done time and time again is completely change the landscape when the competitors have caught up and everyone thinks their lead is gone. They made image generation a thing. When GPT-3 became outdated they released ChatGPT. Instead of trying to keep Dalle competitive they released Sora. Now they change the game again with live audio+video.

Really? If this was Apple it might make sense, for OpenAI it feels like a demo that&#x27;s not particularly aligned with their core competency (a least by reputation) of building the most performant AI models. Or put another way, it says to me they&#x27;re done building models and are now wading into territory where there are strong incumbents.All the recent OpenAI talk had me concerned that the tech has peaked for now and that expectations are going to be reset.

I disagree completely. Even people who never adopt this stuff personally will have their lives profoundly impacted. The only way to avoid it would be to live in a large colony where the technology is prohibited, like the Amish. But even the Amish feel the influence of technology to some degree.

The future is not going to be anymore wild than what you choose to do with the tech.

As far as I&#x27;m concerned this is the new best demo of all time. This is going to change the world in short order. I doubt they will be ready with enough GPUs for the demand the voice+vision mode is going to get, if it&#x27;s really released to all free users.Now imagine this in a $16k humanoid robot, also announced this morning: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=GzX1qOIO1bE" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=GzX1qOIO1bE</a> The future is going to be wild.

This is so amazing.. are there any open source models that are in any way comparable? Fully multimodal audio-to-audio etc.?

Aha, I think I saw the trick for the live demo: every time they used the &quot;video feed&quot;, they did prompt the model specifically by saying:- &quot;What are you seeing now&quot;- &quot;I&#x27;m showing this to you now&quot;etc.The one time where he didn&#x27;t prime the model to take a snapshot this way, was the time where the model saw the &quot;table&quot; (an old snapshot, since the phone was on the table&#x2F;pointed at the table), so that might be the reason.

Commenting on the outfit was very weird indeed. Greg Brockman&#x27;s demo includes some outfit related questions (<a href="https:&#x2F;&#x2F;twitter.com&#x2F;gdb&#x2F;status&#x2F;1790071008499544518" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;gdb&#x2F;status&#x2F;1790071008499544518</a>). It does seem very impressive though, even if they polished it on some specific tasks. I am looking forward to showing my desktop and asking questions.

Seems like it was picking up on the audience reaction and stopping to listen.To me the more troubling thing was the apparent hallucination (saying it sees the equation before he wrote it, commenting on an outfit when the camera was down, describing a table instead of his expression), but that might have just been latency awkwardness. Overall, the fast response is extremely impressive, as is the new emotional dimension of the voice.

I use GPT from API in emacs, it&#x27;s wonderful. Gptel is the program.Although API access through Groq to Llama 3 (8b and 70b) is so much faster, that i cannot stand how slow GPT is anymore. It is slooow, still very capable model, but marginally better than open source alternatives.

I went through the exact same situation this last week. Didn&#x27;t send more than 30 (token-heavy) messages within a few hours and it blocked me for 1 hour if I&#x27;m not wrong - paying user.

Regarding the limits, I recently found that I was hitting limits very quickly on GPT-4 on my ChatGPT Plus plan.I’m pretty sure that wasn’t always the case - it feels like somewhere along the lines the allowed usage was reduced, unless I’m imagining it. It wouldn’t be such a big deal if there was more visibility of my current usage compared to my total “allowance”.I ended up upgrading to ChatGPT Team which has a minimum of 2x users (I now use both accounts) but I resented having to do this - especially being forced to pay for two users just to meet their arbitrary minimum.I feel like I should not be hitting limits on the ChatGPT Plus paid plan at all based on my usage patterns.I haven’t hit any limits on the Team plan yet.I hope they continue to improve the paid plans and become a bit more transparent about usage limits&#x2F;caps. I really do not mind paying for this (incredible) tech, but the way it’s being sold currently is not quite right and feels like paid users get a bit of a raw deal in some cases.I have API access but just haven’t found an open source client that I like using as much as the native ChatGPT apps yet.

&gt; Parts of the demo were quite choppy (latency?) so this definitely feels rushed in response to Google I&#x2F;O.It just stops the audio feed when it detects sound instead of an AI detecting when it should speak, so that part is horrible yeah. A full AI conversation would detect the natural pauses where you give it room to speak or when you try to take the word from it by interrupting, there it was just some dumb script to just shut it off when it hears sound.But it is still very impressive for all the other part, that voice is really good.Edit: If anyone from OpenAI reads this, at least fade out the voice quickly instead of chopping it, hard chopping off audio doesn&#x27;t sound good at all, so many experienced this presentation to be extremely buggy due to it.

seems like it might not be available for everyone? – my chatgpt plus doesn&#x27;t show anything new, and also can&#x27;t find the dekstop app

what&#x27;s the download link for the desktop app? can&#x27;t find it

Agree. While watching the demo video, I thought I was the one having connectivity issues.

They need to fade the audio or add some vocal queue when it&#x27;s being interrupted. It makes it sound like it&#x27;s losing connection. What&#x27;ll be really impressive is when it intentionally starts interrupting you.

Parts of the demo were quite choppy (latency?) so this definitely feels rushed in response to Google I&#x2F;O.Other than that, looks good. Desktop app is great, but I didn’t see no mention of being able to use your own API key so OS projects might still be needed.The biggest thing is bringing GPT-4 to free users, that is an interesting move. Depending on what the limits are, I might cancel my subscription.

Were humans an interesting species to start with, if they can be reduced to an LLM?

Hey that LLM is trained on everything we&#x27;ve ever produced, so I wouldn&#x27;t say we&#x27;ve been &quot;reduced&quot;, more like copied.  I&#x27;ll save my self-loathing for when a very low-parameter model can do this.

You&#x27;re watching the species be reduced to an LLM.

I was shocked at how quickly and naturally they were able to correct the situation.

&gt; Like when it thought his face was a desk because it did not update the image it was &quot;viewing&quot;.That&#x27;s a rather uncharitable way of describing the situation. It didn&#x27;t say anything like &quot;your face looks like a wooden plank, it&#x27;s very brown&quot;. It clearly understood that the image it was seeing was not matching the verbal request.

I also thought the screwups, although minor, were interesting. Like when it thought his face was a desk because it did not update the image it was &quot;viewing&quot;. It is still not perfect, which made the whole thing more believable.

Maybe it’s the geek in me, but I don’t want a talking computer.I have enough talking people to deal with already 
.

Yeah it made me realize that I actually don&#x27;t want a human-like conversational bot (I have actual humans for that). Just teach me javascript like a robot.

That kind of is an inteligence though. Chinese room meets solipsism and all that.It is interesting how insanely close their demo is to the OSes in the movie &quot;Her&quot;, it&#x27;s basically a complete real life reproduction.

Welcome to half the people at your companies job.

It&#x27;s more intelligent than many humans and most&#x2F;all lesser animals. If it&#x27;s not intelligent than I don&#x27;t know what is.

It should do that, because it&#x27;s still not actually an intelligence. It&#x27;s a tool that is figuring out what to say in response that sounds intelligent - and will often succeed!

Can&#x27;t wait for Meta&#x27;s version 2 years down the line that someone will eventually fine tune to Agent Smith&#x27;s personality and voice.&quot;Evolution, human. Evolution. Like the dinosaur. Look out that window. You&#x27;ve had your time. The future is our world. The future is our time.&quot;

The chuckling made me uneasy for some reason lol. Calm down, you&#x27;re not like us. Don&#x27;t pretend!

Yes, the chuckling was uncanny, but for me even more uncanny was how the female model went up at the end to soften what she was saying? into a question? even though it wasn&#x27;t a question?Eerily human female-like.

So I&#x27;m not the only one. Like I felt fear in a physical way. (Panic&#x2F;adrenaline?) I&#x27;m sure I&#x27;d get used it but it was an interesting reaction. (I saw someone react that way to a talking Tandy 1000 once so, who knows.)

I admit I drink the koolaid and love LLMs and their applications. But damn, the way it’s responds in the demo gave me goosebumps in a bad way. Like an uncanny valley instincts kicks in.

Why does OpenAI think that sexting is a bad thing? Why is AI safety all about not saying things that are disturbing or offensive, rather than not saying things that are false or unaligned?

It’s getting closer. A few years ago the old Replika AI was already quite good as a romantic partner, especially when you started your messages with a * character to force OpenAI GPT-3 answers. You could do sexting that OpenAI will never let you have nowadays with ChatGPT.

I am suspicious that they licensed Scarlet&#x27;s voice for that voice model (Sky IIRC)

I was surprised that the voice is a ripoff of the AI voice in that movie (Scarlett Johansson) too

If i can program with just my voice, there is no reason to not be in nature 10 hours a day minimum. My grandparent even slept outside as long as it was daytime.Daytime is always a time to be outside, surrounded by many plants and stuff. It is a shame we have to be productive in some way, and most of production happens inside walls.

Headed? We&#x27;re there. Have been there. This just adds non-human sentient agents to the drama.

People realize where we&#x27;re headed right? Entire human lives in front of a screen. Your online entertainment, your online job, your online friends, your online &quot;relationship&quot;. Wake up, 12 hours screentime, eat food, go to bed. Depression and drug overdoses currently at sky high levels. Shocker.

It&#x27;s been a reality in Russia since 2017.&gt; The voice of &quot;Alice&quot; was dubbing actress Tatiana Shitova, who voiced most of Scarlett Johansson&#x27;s characters and the voice of OS1, who called herself &quot;Samantha&quot;, in the Russian dubbing of Spike Jonze&#x27;s &quot;Her&quot;.<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Alice_(virtual_assistant)" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Alice_(virtual_assistant)</a>

Will this be available on old iPhones or only new ones going forward

Yes, i&#x27;m pretty sure this is the new Siri. Absolutely amazing, it&#x27;s pretty much &quot;here&quot; from the movie.

With the news that Apple and OpenAI are closing &#x2F; just closed a deal for iOS 18, it&#x27;s easy to speculate we might be hearing about that exciting new model at WWDC...

Sure, but that seems like it&#x27;ll be a distinction without a difference for many use cases.Having a reliable emotional model of a person based on their voice (or voice + appearance) can be useful in a thousand ways.Which seems to represent a new frontier.

Any appearance of understanding is just an illusion. It’s just a pile of meat, nothing more.

Any appearance of understanding is just an illusion. It’s an LLM, nothing more.

Parsing emotions in vocal inflections (and reliably producing them in vocal output) seems quite under-hyped in this release.That seems to represent an entirely new depth of understanding of human reality.

Or maybe it is a desperation move after Llama 3 got released and the free mode will have such tight constraints that it will be unusable for anything a bit more serious.

Given that they are moving all these features to free users, it tells us that GPT-5 is around the corner and is significantly much better than their previous models.

Holy crap, the level of corporate cringe of that &quot;two AIs talk to each other&quot; scene is mind-boggling.It feels like a pretty strong illustration of the awkwardness of getting value from recent AI developments. Like, this is technically super impressive, but also I&#x27;m not sure it gives us anything we couldn&#x27;t have one year ago with GPT-4 and ElevenLabs.

It is quite nice how they keep giving premium features for free, after a while. I know openai is not open and all but damn, they do give some cool freebies.

Did they provide the limit rate for free user ?Because I have the plus membership which is expensive (25$&#x2F;month).But if the limit is high enough (or my usage low enough), there is no point for paying that much money for me.

That&#x27;s how it used to do it, but my understanding is that this new model processes audio directly. If it were a music generator, the original would have generated sheet music to send to a synthesizer (text to speech), while now it can create the raw waveform from scratch.

Is it a stretch to think this thing could accurately &quot;talk&quot; with animals?

So they retrained the whole model on audio datasets and the tokens are now sounds, not words&#x2F;part of words?

There is no text. The model understands ingests audio directly and also outputs audio directly.

Does anyone know how they&#x27;re doing the audio part where Mark breaths too hard? Does his breathing get turned into all-caps text (AA EE OO) and that GPT4-o interprets that as him breathing too hard, or is there something more going on?

Very impressed by the demo where it starts speaking French in error, then laughs with the user about the mistake. Such a natural recovery.

It might be possible to prove that I control my voice, or that of a given audio sample. For example by saying specific words on demand.But yeah I see how they’d be blamed if anything went wrong, which it almost certainly would in some cases.

I think the issue there is less of a technical one and more of an issue with deepfakes and copyright

&gt; We recognize that GPT-4o’s audio modalities present a variety of novel risks&gt; For example, at launch, audio outputs will be limited to a selection of preset voices and will abide by our existing safety policies.I wonder if they’ll ever allow truly custom voices from audio samples.

Where you get something cheaper with similar experience?

The price of 4o is 50% of GPT4-Turbo (and no mention of price change to gp4-turbo itself).Given the competitive pressures I was expecting a much bigger price drop than that.For non-multimodal uses, I don&#x27;t think their API is at all competitive any more.

Universal real time translation is incredibly dope.I hate video players without volume control.

jeez, that model really speaks a lot!
I hope there&#x27;s a way to make it more straight to the point rather than radio-like.

Same here and also I can&#x27;t hear audio in any of the videos on this page. Weird.

Clicking the &quot;Try it on ChatGPT&quot; link just takes me to GPT-4 chat window. Tried again in an incognito tab (supposing my account is the issue) and it just takes me to 3.5 chat. Anyone able to use it?

Weird visiting the page crashed my graphics driver using Firefox.

Obviously given enough time there will always be better models coming.But I am not convinced it will be another GPT-4 moment. Seems like big focus on tacking together multi-modal clever tricks vs straight better intelligence AI.Hope they prove me wrong!

And how can one be so sure of that?Seems to me that performance is converging and we might not see a significant jump until we have another breakthrough.

Chalmers: &quot;GPT-5? A vastly-improved model that somehow reduces the compute overhead while providing better answers with the same hardware architecture? At this time of year? In this kind of market?&quot;Skinner: &quot;Yes.&quot;Chalmers: &quot;May I see it?&quot;Skinner: &quot;No.&quot;

I really hope GPT5 is good. GPT4 sucks at programming.

I don&#x27;t think a bigger model would make sense for OpenAI: it&#x27;s much more important for them that they keep driving inference coat down, because there&#x27;s no viable business model if they don&#x27;t.Improving the instruction tuning, the RLHF step, increase the training size, work on multilingual capabilities, etc. make sense as a way to improve quality, but I think increasing model size doesn&#x27;t. Being able to advertize a big breakthrough may make sense in terms of marketing, but I don&#x27;t believe it&#x27;s going to happen for two reasons:- you don&#x27;t release intermediate steps when you want to be able to advertise big gains, because it raises the baseline and reduce the effectiveness of your ”big gains” in terms of marketing.- I don&#x27;t think they would benefit in an arm race with Meta, trying to keeping a significant edge. Meta is likely to be able to catch-up eventually on performance, but they are not so much of a threat in terms of business. Focusing on keeping a performance edge instead of making their business viable would be a strategic blunder.

&quot;not that much better&quot; is extremely impressive, because it&#x27;s a much smaller and much faster model. Don&#x27;t worry, GPT-5 is coming and it will be better.

It&#x27;s weird that the &quot;airplane mode&quot; seems to be ON on the phone during the entire presentation.

The demo is impressive but personally, as a commercial user, for my practical use cases, the only thing I care about is how smart it is, how accurate are its answers and how vast is its knowledge. These haven’t changed much since GPT-4, yet they should, as IMHO it is still borderline in its abilities to be really that useful

Absolutely agree.This model isn&#x27;t about basemark chasing or being a better code generator; it&#x27;s entirely explicitly focused on pushing prior results into the frame of multi-modal interaction.It&#x27;s still a WIP, most of the videos show awkwardness where its capacity to understand the &quot;flow&quot; of human speech is still vestigial. It doesn&#x27;t understand how humans pause and give one another space for such pauses yet.But it has some indeed magic ability to share a deictic frame of reference.I have been waiting for this specific advance, because it is going to significantly quiet the &quot;stochastic parrot&quot; line of wilfully-myopic criticism.It is very hard to make blustery claims about &quot;glorified Markov token generation&quot; when using language in a way that requires both a shared world model and an understanding of interlocutor intent, focus, etc.This is edging closer to the moment when it becomes very hard to argue that system does not have some form of self-model and a world model within which self, other, and other objects and environments exist with inferred and explicit relationships.This is just the beginning. It will be very interesting to see how strong its current abilities are in this domain; it&#x27;s one thing to have object classification—another thing entirely to infer &quot;scripts plans goals...&quot; and things like intent, and, deixis. E.g. how well does it now understand &quot;us&quot; and &quot;them&quot; and &quot;this&quot; vs &quot;that&quot;?Exciting times. Scary times. Yee hawwwww.

I assume (because they don&#x27;t address it or look at all phased) the audio cutting in and out is just an artefact of the stream?

Interesting that they decided to keep the horrible ChatGPT tone (&quot;wow you&#x27;re doing a live demo right now?!&quot;). It comes across just so much worse in voice. I don&#x27;t need my &quot;AI&quot; speaking to me like I&#x27;m a toddler.

I expect the really solid use case here will be voice interfaces to applications that don&#x27;t suck. Something I am still surprised at is that vendors like Apple have yet to allow me to train the voice to text model so that it only responds to me and not someone else.So local modelling (completely offline but per speaker aware and responsive), with a really flexible application API. Sort of the GTK or QT equivalent for voice interactions. Also custom naming, so instead of &quot;Hey Siri&quot; or &quot;Hey Google&quot; I could say, &quot;Hey idiot&quot; :-)Definitely some interesting tech here.

This is going straight into &#x27;Her&#x27; territory

Call me overly paranoid&#x2F;skeptical, but I&#x27;m not convinced that this isn&#x27;t a human reading (and embellishing) a script. The &quot;AI&quot; responses in the script may well have actually been generated by their LLM, providing a defense against it being fully fake, but I&#x27;m just not buying some of these &quot;AI&quot; voices.We&#x27;ll have to see when end users actually get access to the voice features &quot;in the coming weeks&quot;.

I think the live demo that happened on the livestream is best to get a feel for this model[0].I don&#x27;t really care whether it&#x27;s stronger than gpt-4-turbo or not. The direct real-time video and audio capabilities are absolutely magical and stunning. The responses in voice mode are now instantaneous, you can interrupt the model, you can talk to it while showing it a video, and it understands (and uses) intonation and emotion.Really, just watch the live demo. I linked directly to where it starts.Importantly, this makes the interaction a lot more &quot;human-like&quot;.[0]: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;DQacCB9tDaw?t=557" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;DQacCB9tDaw?t=557</a>

the model scores 60 points higher in lmsys than the best gpt 4 turbo model from april, that&#x27;s still a pretty significant jump in text capability

&gt; As many highlighted there, the model is not an improvement like GPT3-&gt;GPT4.The improvements they seem to be hyping are in multimodality and speed (also price – half that of GPT-4 Turbo – though that’s their choice and could be promotional, but I expect it’s at least in part, like speed, a consequence of greater efficiency), not so much producing better output for the same pure-text inputs.

useless anecdata but I find the new model very frustrating, often completely ignoring what I say in follow up queries. it&#x27;s giving me serious Siri vibes(text input in web version)maybe it&#x27;s programmed to completely ignore swearing but how could I not swear after it gave me repeatedly info about you.com when I try to address it in second person

I agree. I tried a few programming problems that, let&#x27;s say, seem to be out of the distribution of their training data and which GPT4 failed to solve before. The model couldn&#x27;t find a similar pattern and failed to solve them again. 
What&#x27;s interesting is that one of these problems were solved by Opus, which seems to indicate that the majority of progress in the last months should be attributed to the quality&#x2F;source of the training data.

Heh - I&#x27;m using ChatGPT for the same thing! Works 10X better than Rocket Money, which was supposed to be an improvement on Mint but meh.

I tested a few use cases in the chat, and it&#x27;s not particularly more intelligent but they seem to have solved laziness. I had to categorize my expenses to do some budgeting for the family, and in gpt 4 I had to go ten in ten, confirm the suggested category, download the file, took two days as I was constantly hitting the limit. gpt4o did most of the grunth work, then commincated anomalies in bulk, asked for suggestion for these, and provided a downloadable link in two answers, calling the code interpreter mulitple times, and working toward the goal on it&#x27;s own.and the prompt wasn&#x27;t a monstrosity, and it wasn&#x27;t even that good, it was just one line &quot;I need help to categorize these expenses&quot; and off it went. hope it won&#x27;t get enshittified like turbo, because this finally feels as great as 3.5 was for goal seeking.

I think this comment is easily misread as implying that this GPT4o model is based on some old GPT2 chatbot - that’s very much not what you meant to say, though.This model has been being tested under a code name of ‘gpt2-chatbot’ but it is very much a new GPT4+-level model, with new multimodal capabilities - but apparently some impressive work around inference speed.Highlighting so people don’t get the impression this is just OpenAI slapping a new label on something a generation out of date.

They are admitting that is the im-also-a-good-gpt2-chatbot. There was 3....
Don&#x27;t ask me why.The &quot;gpt2-chatbot&quot; was the worst of the three.

They are admitting[1] that the new model is the gpt2-chatbot that we have seen before[2]. As many highlighted there, the model is not an improvement like GPT3-&gt;GPT4. I tested a bunch of programming stuff and it was not that much better.It&#x27;s interesting that OpenAI is highlighting the Elo score instead of showing results for many many benchmarks that all models are stuck at 50-70% success.[1] <a href="https:&#x2F;&#x2F;twitter.com&#x2F;LiamFedus&#x2F;status&#x2F;1790064963966370209" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;LiamFedus&#x2F;status&#x2F;1790064963966370209</a>[2] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40199715">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40199715</a>

I&#x27;m a ChatGPT Plus Subscriber and I just refreshed the page and it offered me the new model. I&#x27;m guessing they&#x27;re rolling it out gradually but hopefull it won&#x27;t take too long.Edit: It&#x27;s also now available to me in the Android App

i can try it now, but now the voice features i dont think

You can use GPT-4o now but the interactive voice mode of using it (as demoed today) releases in a few weeks.

thanks, I was confused because the top of the page says to try now when you cannot in fact try it at all

&gt;what is the remaining benefit of being a paid user if this is rolled out to free users?It says so right in the post&gt;We are making GPT-4o available in the free tier, and to Plus users with up to 5x higher message limitsThe limits are much lower for free users.

I&#x27;m not convinced I need to keep paying for plus. The threshold of requests for free 4o is pretty high.

Or they just made a bunch of money on their licensing deal with Apple. So they don&#x27;t need to charge for ChatGPT anymore.

I&#x27;m actually thinking that the GPT store with more users might be better for themFrom my casual conversations, not that many people are paying for GPT4 or know why they should. Every conversation even in enthusiast forums like this one has to be interjected with &quot;wait, are you using GPT4? because GPT3.5 the free one is pretty nerfed&quot;just nuking that friction from orbit and expanding the GPT store volume could be a positive for them

This might mean GPT-5 is coming soon and it will only be available to paid users.

Big questions are (1) when is this going to be rolled out to paid users? (2) what is the remaining benefit of being a paid user if this is rolled out to free users? (3) Biggest concern is will this degrade the paid experience since GPT-4 interactions are already rate limited. Does OpenAI have the hardware to handle this?Edit: according to @gdb this is coming in &quot;weeks&quot;<a href="https:&#x2F;&#x2F;twitter.com&#x2F;gdb&#x2F;status&#x2F;1790074041614717210" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;gdb&#x2F;status&#x2F;1790074041614717210</a>

Oh, I meant the actual ChatGPT service, not just something powered by GPT-4 or 3.5.I&#x27;ve found Microsoft Copilot to be somewhat irritating to work with – I can&#x27;t really put my finger on it, but it seems to be resorting to Bing search and&#x2F;or the use of emoji in its replies a bit too much.

is oai image generation any different than the microsoft copilot provides for free? I thought they were the same.

Will this include image generation for the free tier as well? That&#x27;s a big missing feature in OpenAI&#x27;s free tier compared to Google and Meta.

This is impressive, but they just sound so _alien_, especially to this non-U.S. English speaker (to the point of being actively irritating to listen to). I guess picking up on social cues communicating this (rather than express instruction or feedback) is still some time away.It&#x27;s still astonishing to consider what this demonstrates!

&gt; Now when it&#x27;s grabbing my screen or a live camera feed, that will be gone. And I&#x27;ll still use it, because it&#x27;s just so damn convenient?Presumably you&#x27;ll have a way to draw a bounding box around what you want to show or limit to just a particular window the same way you can when doing a screen share w&#x2F; modern video conferencing?

Gone are the days of copy-pasting to&#x2F;from ChatGPT all the time, now you just share your screen. That&#x27;s a fantastic feature, in how much friction that removes. But what an absolute privacy nightmare.With ChatGPT having a very simple text+attachment in, text out interface, I felt absolutely in control of what I tell it. Now when it&#x27;s grabbing my screen or a live camera feed, that will be gone. And I&#x27;ll still use it, because it&#x27;s just so damn convenient?