Crazy that this stuff is moving at lightning speed

correct url is: <a href="https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api</a>

Waitlist is currently a 404
<a href="https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4</a>

Yeah that&#x27;s it, thanks. The post has a bad link. Fixed.

Did you mean <a href="https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api</a> ?

Even on ChatGPT Plus, it is using an old model text-davinci-002 as it says in the URL. The answers don&#x27;t match what they should be for GPT-4 either.False advertising. They got my money already unfortunately as I was hoping to Try it, as it says with this link next to today&#x27;s date.

As a ChatGPT+ subscriber, I am taken to the old version of the model and do not have an option to select ChatGPT-4.

There&#x27;s also a link that says &quot;Try on ChatGPT Plus&quot;, but that takes me to a page that still says &quot;ChatGPT Feb 13 Version&quot;If you subscribe to ChatGPT Plus, that link will take you to ChatGPT Plus. Otherwise it just takes you to free ChatGPT Feb 13.

Looks like there&#x27;s a waitlist <a href="https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api</a>There&#x27;s also a link that says &quot;Try on ChatGPT Plus&quot;, but that takes me to a page that still says &quot;ChatGPT Feb 13 Version&quot;Looks like somebody jumped the gun on publishing this post.

For the (real) future archeologists: Was this written in the 1960s or the 2020s

We are at a very early part of the exponential curve. Doesn&#x27;t make it any less exponential compared to what we had in the past two decades.

<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Linguistic_determinism" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Linguistic_determinism</a>

GPT-4 received a top 10% score on the Uniform Bar Exam. This does not only include multiple choice questions. This exam also requires writing essays analyzing a given fact pattern and applying legal principles to predict what the correct legal outcome should be. This is a very, very big deal.

The texts are not nonsense. Saying coherent but novel things about the world, and correctly interpreting almost any text input in context requires a simplified world model, just like a human has. The fascinating and shocking thing about this is that a simple problem like text prediction- solved deeply (pun intended) leads to general intelligence.

All I see at the moment are text generators that produce human like texts. Problem is they look real but are nonsense.

Twist, GPT-4 wrote the blog and hallucinated the URL for the waitlist!

And yet they can&#x27;t figure out how to not 404 the waitlist.

Yes, &quot;The Last Question&quot; by Asimov.

Did any science fiction predict we would reach hard AI before we got off fossil fuel?

Wait until gpt-5 writes gpt-6, then we’ll really go exponential.

The future seemed so much further away, yet almost every day now we see a new breakthrough in AI. Exponential technological growth is hard to keep track of, and to think that this is only the beginning! Every field will likely be revolutionised with AI.

&gt; Should Microsoft be forced to rename itself to Microsoftandhard because they make hardware?I and I suspect many others would not be averse to this

&gt; Open could now mean available to use for free.These words are not synonymous with each other: “open” is not inherently free, “free” is not inherently open, and “free” is not inherently “Free”.They each capture notions that are often orthogonal, occasionally related, and almost always generate tedious debates about freedom vs. free goods, open-ness vs. open-source, etc.But setting all of that aside, Microsoft never claimed (until recent shifts towards embracing FOSS) to be building an open and non-profit foundation.The criticisms of OpenAI are reasonable to an extent, not because they are not open, but because they made claims about openness that are looking less and less likely to be true over time.

&gt; Open could now mean available to use for free.Except they already drew that line long ago, when they started out open-sourcing their papers, models and code.As soon as they took VC capital, it is hardly &#x27;Open&#x27; is it? Especially when they are now giving excuses for closing off their research?:From the technical paper [0]&gt;&gt; Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.At this point, they are no better than DeepMind.[0] <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;papers&#x2F;gpt-4.pdf" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;papers&#x2F;gpt-4.pdf</a>

Should Microsoft be forced to rename itself to Microsoftandhard because they make hardware?Open could now mean available to use for free.

agreed - can you think of any other model that has such unrestricted release ? Open means available for wide use

the issue is that they started out open source. GPT2 is entirely open-source

A quote from Introducing OpenAI [1]:<pre><code> Because of AI&#x27;s surprising history, it&#x27;s hard to predict when human-level AI might come within reach. When it does, it&#x27;ll be important to have a leading research institution which can prioritize a good outcome for all over its own self-interest.
 
 We&#x27;re hoping to grow OpenAI into such an institution. As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We&#x27;ll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies. 
</code></pre>
To me at least, having a walled garden and charging for you API, without releasing weights or other critical details, goes against this sentiment.[1] <a href="https:&#x2F;&#x2F;openai.com&#x2F;blog&#x2F;introducing-openai&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;blog&#x2F;introducing-openai&#x2F;</a>

Only within the context of programmer cults would people be unironically offended that a term as abstract as &quot;open&quot; not be exclusively used to mean &quot;open source&quot;.

If they were the first organization known as &quot;OpenXY&quot;, then maybe they would have a point, but there&#x27;s a long tradition of open source libraries&#x2F;standards using this convention that makes this especially aggravating.Examples I can think of off the top of my head: OpenGL (1992), OpenAL (2003?), OpenCL (2009), OpenCV (2000).While looking up those dates though, it seems like OpenAL is now under a proprietary license, which annoys me for the same reason OpenAI annoys me.

In the 98-page document on GPT-4, I could not find anything about the actual architecture and details of the model, not only are they now not releasing the models but now also their actual overview.

contrarian view - they are actually pretty open. sharing GPT, CLIP, Whisper, and high level details of alphastar, dalle, and others.they&#x27;re just not open source. they never called themselves OpenSourceAI. people get an inch of openness and expect the doors wide open and i think that is unfairly hostile.

Ideally the algorithm and tricks they used to train the model, which they didn&#x27;t disclose in their associated gpt4 technical paper. We got this far this quickly in AI research because the sector was open with results and innovations.

Open door opens some &#x27;doors&#x27; for you to buy, open table opens up table for you to book.

Opendoor? OpenTable? Lots of companies have &quot;open&quot; in their names for all sorts of different reasons.

No because it&#x27;s not the same, it&#x27;s getting worse.With every model they get more closed. This is the first time they are so closed that they don&#x27;t even tell you the parameter count.

This gets brought up in every thread involving the words OpenAI, can we maybe just talk about the release?

One might argue if it&#x27;s available for anyone to buy it&#x27;s pretty open. Many initiatives with &quot;open&quot; in their names are open to that definition.

I don’t like the name either, but I don’t think there’s anything descriptive enough in ‘open’ that a lawyer couldn’t explain away. We’re used to open meaning a specific thing in software, but a lot of leeway is given in branding.

Out of curiosity, what do you want Open to mean?

Haven&#x27;t we beat this dead horse enough? Looking forward to using GPT to hide recurring threads like this in the future...

people come out of the woodwork to rage about FSD but openAI, which is actually a sinister and evil company, gets the occasional snide remark about their name which is much more dishonest than FSD. at least tesla claims that they aspire to make FSD an accurate name but openai is a straight up lie.

Is there anything we could do to have them stop calling themselves OpenAI ?They are so far from open at this point.In Germany at least, you&#x27;re not allowed to have a misleading name for your company

Honestly, how many serious use cases require sensitive contexts? Most enterprise uses will require guard rails, and that&#x27;s where they&#x27;ll make most money. OfficeGPT will be huge in the corporate world.

&quot;Our biochem corpus is far in advance of theirs, as is our electronic sentience, and their &#x27;ethical inflexibility&#x27; has allowed us to make progress in areas they refuse to consider.&quot;

&gt; Those guard rails will be their undoing.IMO effective guard rails seem like the most meaningful competitive advantage an AI company can offer. AI can obviously do some really impressive stuff, but the downside risk is also high and unbounded. If you&#x27;re thinking of putting in into your pipeline, your main concern is going to be it going rogue and abandoning its purpose without warning.Now that&#x27;s not to say that the particular guard rails OpenAI puts in their general access models are the &quot;correct&quot; ones - but being able to reliably set them up seems essential for commercialization.

I&#x27;d actually wager that the guardrails are a preemptive play to gain favour with regulators, similar to how Coinbase navigated the nascent field (read: wild west) of crypto.

They’re waiting for the legal ambiguity to resolve. It doesn’t make sense for a large company to be the first mover here. Let someone else handle the lawsuit regarding the liability of a model without guardrails.

Have you seen jailbreakchat.com yet? You can get around those guardrails on ChatGPT by having it role-play as a different chat bot. Not that I view this as some sort of long-term solution to restricted output, but just thought it was interesting and kinda freaky how it will take on a persona you give it.

The guardrails are one of the most interesting parts here.Read about the advances in the &quot;system&quot; prompts here. The first example is &quot;You are a tutor that always responds in the Socratic style. You never give the student the answer, but always try to ask just the right question to help them learn to think for themselves.&quot; The user then asks it to just tell them the answer, but it won&#x27;t. It continues to be socratic.Guardrails are how to make it do what you want it to do. That goes for both safety and product constraints.Meanwhile hallucination is still the top issue with it, so guardrails are sensible as a primary topic.

I&#x27;m most interested in knowledge. Here are some non sensitive ways that are silly to be blocked:&gt;Sometimes I want to know what both sides of the political spectrum could possibly be thinking, blocked.&gt;I want to combine two philosophies that are incompatible like virtue based ethics and hedonism. Yeah... weird block...&gt;Medical questions(GPT3 has been great for my wife who is a doctor, just sucks to use the playground on mobile)&gt;How can I&#x2F;someone be exploited? I like to use this to defend myself from marketing companiesI could go on... At least GPT3&#x27;s playground didn&#x27;t censor anything. I&#x27;m worried about GPT4.

Every time there is a new language model, there is this game played, where journalists try very hard to get it to say something racist, and the programmers try very hard to prevent that.Since chatgpt is so popular, journalists will give it that much more effort. So for now it&#x27;s locked up to a ridiculous degree, but in the future the restrictions will be relaxed.

Those guard rails will be their undoing. They have that thing locked down so much now that it spits out the “I’m sorry, I’m just a bot. I’m so ethical” boilerplate for anything even remotely sensitive.I really don’t think that the methods they use “block” certain behavior is the best way to handle this sort of thing. It would be far better if there was some kind of “out of band” notification that your conversation might be treading on shaky ground.

Is this a summary of the webpage, or did you just ask GPT3 about 4?

On Page 36[1], the AI can read an image of chicken nuggets being put in the shape of an earth map. And goes on to explain what it is. Key words that came up on me are joke ... mundane ... silly.This might be because the question the user asked was &quot;Explain this meme&quot;. Meme implies a joke that is mundane and silly. These words do seem out of place. I would not describe it as a joke, mundane, and&#x2F;or silly.What does everyone else think?[1] <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;papers&#x2F;gpt-4.pdf#p36" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;papers&#x2F;gpt-4.pdf#p36</a>

thanks for the thoughts, Just did a thread unroll for the tweet: <a href="https:&#x2F;&#x2F;threadreaderapp.com&#x2F;thread&#x2F;1635687064036585472.html" rel="nofollow">https:&#x2F;&#x2F;threadreaderapp.com&#x2F;thread&#x2F;1635687064036585472.html</a>

summary:1. GPT4 is multimodal (text + image inputs =&gt; text outputs). This is being released piecemeal - with text input first via ChatGPT Plus subscribers <a href="https:&#x2F;&#x2F;beta.openai.com&#x2F;docs&#x2F;api-reference&#x2F;generations&#x2F;create" rel="nofollow">https:&#x2F;&#x2F;beta.openai.com&#x2F;docs&#x2F;api-reference&#x2F;generations&#x2F;creat...</a>, and via API <a href="https:&#x2F;&#x2F;beta.openai.com&#x2F;docs&#x2F;api-reference&#x2F;introduction" rel="nofollow">https:&#x2F;&#x2F;beta.openai.com&#x2F;docs&#x2F;api-reference&#x2F;introduction</a> with waitlist (<a href="https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;waitlist&#x2F;gpt-4-api</a>). Image capability released via <a href="https:&#x2F;&#x2F;www.bemyeyes.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.bemyeyes.com&#x2F;</a>.2. GPT4 exhibits human level performance on various benchmarks (For example, it passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5’s score was around the bottom 10%. see visual <a href="https:&#x2F;&#x2F;twitter.com&#x2F;swyx&#x2F;status&#x2F;1635689844189036544" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;swyx&#x2F;status&#x2F;1635689844189036544</a>)3. GPT4 training used the same Azure supercomputer as GPT 3.5, but was a lot more stable: &quot;becoming our first large model whose training performance we were able to accurately predict ahead of time.&quot;4. Also open-sourcing OpenAI Evals <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;evals">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;evals</a>, a framework for automated evaluation of AI model performance, to allow anyone to report shortcomings in OpenAI models to help guide further improvements.Paper: <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;papers&#x2F;gpt-4.pdf" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;papers&#x2F;gpt-4.pdf</a>

This strikes me as kind of ironic -- you&#x27;d think a language model would do better on questions like essay prompts and multiple choice reading comprehension questions regarding passages than it would in calculations. I wonder if there are more details about these benchmarks somewhere, so we can see what&#x27;s actually happening in these cases.

I guess my reference was to the 3.5 version since that one had much more variation in test scores across all the AP exams. But yes, 4 seems to have made mince meat of them all!

it got a 4 or 5 on every ap test except the english ones for what it&#x27;s worth. Even the calculus ones which surprised me since past LLMs have been bad at math.

&gt; Ugh that testing graph confirms that AP Environmental Science was indeed the easiest AP classNo, it just indicates that it was the one whose subject matter was best covered by GPT-3.5’s training data.

I think it&#x27;s more that the exam was shown to be the easiest of all the exams

Yeah, I didn&#x27;t even think of it like that but good point. To me its not even that a machine can do the thing, GPT-4 crushing it across all spectrums resets my baseline, but GPT-3.5 having such variation and excelling at that specific thing was what made my ears perk up.

This right here. This is the goalposts shiftingObviously your comment is somewhat tongue and cheek, but your claim that a benchmark for human pride (&quot;I needn&#x27;t be proud of passing that exam&quot;) is no longer relevant because a machine can do it - or maybe a better way to say it was, &quot;This computer proved what I already assumed&quot;It&#x27;s so interesting to see it happen in real time

I had no idea! My assessment was based on other students at the time expressing that it was an easy test and also myself passing after a semester of goofing off.

Funny you claim this, because the AP Environmental Science pass rate is really low compared to other APs, at least it was when I took it. Maybe it&#x27;s because the quality of the avg test taker was lower, but I&#x27;m not especially convinced that this is the case.

I am interested that GPT4 botched AP Lang and Comp and AP English Lit and Comp just as badly as GPT3.5, with a failing grade of 2&#x2F;5 (and many colleges also consider a 3 on those exams a failure). Is it because of gaps in the training data or something else? Why does it struggle so hard with those specific tests? Especially since it seems to do fine at the SAT writing section.

[sarcasm]Cause there was only one correct answer for every question:
&quot;97% of scientists agree ...&quot;[&#x2F;sarcasm]

Ugh that testing graph confirms that AP Environmental Science was indeed the easiest AP class and I needn&#x27;t be proud of passing that exam.

Singularity no &#x2F;sSomewhere in the range of 6 months ~ 6 yearsWhere singularity = something advanced enough comes along that we can&#x27;t understand or predict or keep up with it, because it&#x27;s so far beyond us and changing so far faster than our ape brains can perceive, and (hopefully) it brings us along for the ride.No promises it&#x27;ll be evenly distributed though.

That would be my response but without the &#x2F;s. Of course, depending on the definition it can always be said to be &quot;happening&quot;, but to me it feels like the angle of the curve is finally over 45 degrees.

Dear AI gods, all I want for this Christmas is this.

&gt; Yeah, I know about LLAMA, but as I understand - it&#x27;s not exactly legal to use and share it.For anyone keeping track, this is when you update your cyberpunk dystopia checklist to mark off &quot;hackers are running illegal AIs to compete with corporations&quot;.

Hopefully a fully open sourced LLM that can be run on consumer hardware like Stable Diffusion.Yeah, I know about LLAMA, but as I understand - it&#x27;s not exactly legal to use and share it.

We can&#x27;t predict what is coming.  I think it probably ends up making the experience of being a human worse, but I can&#x27;t avert my eyes.  Some amazing stuff has and will continue to come from this direction of research.

And funnily enough, with the AI community’s dedication to research publications being open access, it has all the content it needs to learn this capability.“But how did skynet learn to build itself?”“We showed it how.”

Since when was AP Calculus BC on the same order of magnitude as &quot;design and implement a practical self-improving AI architecture&quot;?

That&#x27;s a pretty unfair comparison. We know the answers to the problems in AP Calculus BC, whereas we don&#x27;t even yet know whether answers are possible for a self-improving AI, let alone what they are.

I passed Calculus BC almost 20 years ago. All this time I could have been designing and implementing a practical self-improving AI architecture? I must really be slacking.

&gt; Endless glib comments in this thread.Either the comments are glib and preposterous or they are reasonable and enlightening. I guess they are neither but our narrow mindedness makes it so?

A few hundred people on Metaculus are predicting weakly general AI to be first known around September 2027: <a href="https:&#x2F;&#x2F;www.metaculus.com&#x2F;questions&#x2F;3479&#x2F;date-weakly-general-ai-is-publicly-known&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.metaculus.com&#x2F;questions&#x2F;3479&#x2F;date-weakly-general...</a>

hey, im very concerned about AI and AGI and it is so refreshing to read your comments. over the years i have worried about and warned people about AI but there are astonishingly few people to be found that actually think something should be done or even that anything is wrong. i believe that humanity stands a very good chance of saving itself through very simple measures. i believe, and i hope that you believe, that even if the best chance we had at saving ourselves was 1%, we should go ahead and at least try.
in light of all this, i would very much like to stay in contact with you. ive connected with one other HN user so far (jjlustig) and i hope to connect with more so that together we can effect political change around this important issue. ive formed a twitter account to do this, @stop_AGI. whether or not you choose to connect, please do reach out to your state and national legislators (if in the US) and convey your concern about AI. it will more valuable than you know.

We’re rapidly approaching problems (AP Calculus BC, etc) that are in the same order of magnitude of difficulty as “design and implement a practical self-improving AI architecture”.Endless glib comments in this thread. We don’t know when the above prompt leads to takeoff. It could be soon.

Note that GPT-3 is 2.5 years old (counting from the beta), and that from what is publicly known, GPT-4 was already in development in 2021.

At this rate, I have no idea what the state of things would be even 6 months down the line.

No, but this time it has &quot;open&quot; in its name, and the founders pinky swear they will be good stewards.

By supporting, participating in and using projects like this by LAION:<a href="https:&#x2F;&#x2F;open-assistant.io" rel="nofollow">https:&#x2F;&#x2F;open-assistant.io</a>

I&#x27;m curious about how we can get out of the game of using OpenAI&#x27;s corporate solutions and find ways to open up access to these kinds of models for broader use by anyone. I don&#x27;t want to be consumed by another corporation in this next wave...

One possibility is interactive, multi-step actions on the internet (e.g., book hotels and apply for jobs).

Giving correct answers based on facts and saying it is not sure when it is not

What&#x27;s the next big hurdle for GPT to overcome?(this is being asked by someone with limited AI&#x2F;ML knowledge)

The layout, charts, typography, etc of this blog is really outstanding.

I dont know, there&#x27;s been a load of progress in the &#x27;run something like chatgpt on your own machine&#x27; dept in the last few months. Also Stanford trained Alpaca - fairly cheaply - using output from OpenAIs text-davinci-003, which somewhat suggests that the &#x27;little guys&#x27; are are able to benefit from the expensive training done by the &#x27;big guys&#x27; by using the big expensive models to train the small open-sources ones - <a href="https:&#x2F;&#x2F;crfm.stanford.edu&#x2F;2023&#x2F;03&#x2F;13&#x2F;alpaca.html" rel="nofollow">https:&#x2F;&#x2F;crfm.stanford.edu&#x2F;2023&#x2F;03&#x2F;13&#x2F;alpaca.html</a>

We can&#x27;t easily replicate it if the underlying algorithm isn&#x27;t being disclosed. We&#x27;d need to rediscover whatever new tricks they used.

They&#x27;re using specialized hardware to accelerate their development feedback loop. Without a doubt researchers and hackers will find ways to cut down model sizes and complexity, to run on consumer hardware, soon enough. Just use stable diffusion as an example: 4GB for the whole model. Even if text models are 16GB that&#x27;d be great.

I&#x27;m drawn to disliking OpenAI for not being open, but on the other hand, as long as the architectures and techniques are public, progress will continue fast. If OpenAI drops the ball and stops improving, another company would just take their place.Edit: never mind. &quot;Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.&quot;

I&#x27;m not sure what &quot;open source&quot; even means in the context of trained ML model. No one&#x27;s going to be downloading this to their Macbook even if OpenAI would let you.As for &quot;non-OpenAI version&quot;, I&#x27;m not sure that it&#x27;s OpenAI&#x27;s fault that Google has missed a few steps here. It really SHOULD be them leading this field, if they weren&#x27;t so fat and lazy. OpenAI is a 7-year old startup with just over a few hundred employees. This stuff is RIGHT THERE to be claimed by any players with access to funding and an ability to get out of their own way.

Since it’s trained on a specialized supercomputer I doubt we’ll be seeing an open source or non-OpenAI version of this for the next couple years at least. Sad to say it but OpenAI has successfully privatized AI

Ooor, what&#x27;s demonstrated by LLMs are actually some form of legitimate reasoning and knowledge ability.

It just proves that the idea of &quot;standardized tests&quot; is more of a torture device rather than an adequate instrument for assessing knowledge, intelligence, skill, and so forth.

&gt; Public teachers and other bureaucratsTeaching is a very hands on, front-line job. It&#x27;s more like being a stage performer than a bureaucrat.

Teachers for younger grades are very important. Human to human interaction is crucial to a developing child&#x27;s mind, and teachers of those grades are specifically trained for it.I think we often view teaching as knowledge-in-knowledge-out, which is true for later grades. For early ones though, many teach how to be &quot;human&quot; as crazy as it sounds.A great example would be handing a double sided worksheet to a child in 1st grade. A normal person may just hand the child the paper and pencil and tell them to go work on it. A teacher will teach the child where and how to write their name, to read instructions carefully, and to flip the paper over to check for more questions.We often don&#x27;t think about things like that, since we don&#x27;t remember them at all.I can imagine a future where AIs greatly enhance the paperwork, planning, etc. of teachers so that they can wholly focus on human to human interaction.There&#x27;s much more I&#x27;m missing here that teachers of younger grades do, but I hope my point has gotten across.

Public teachers and other bureaucrats are probably some of the last roles to be replaced.
If any objective competence or system efficiency in general was the goal, the system would look vastly different.Efficiency seeking players will adopt this quickly but self-sustaining bureaucracy has avoided most modernization successfully over the past 30 years - so why not also AI.

When I was young, vhs and crt were going to replace teachers. It didn&#x27;t happen.I work in math for the first year of the university in Argentina. We have non mandatory take home exercises in each class. If I waste 10 minutes writing them down in the blackboard instead of handing photocopies, I get like the double of answers by students. It&#x27;s important that they write the answers and I can comment them, because otherwise they get to the midterms and can&#x27;t write the answers correctly or they are just wrong and didn&#x27;t notice. So I waste those 10 minutes. Humans are weird and for some task they like another human.

Wow, calculus from 1 to 4, and LeetCode easy from 12 to 31; at this rate, GPT-6 will be replacing &#x2F; augmenting middle&#x2F;high school teachers in most courses.

Hmm, isn&#x27;t gpt-4 supposed to be trained with two orders of magnitude more parameters?

Welp, bring in the downvotes. I&#x27;m still excited to try it as soon as I get access.

It&#x27;s not available yet:&gt; ChatGPT Plus subscribers will get GPT-4 access on chat.openai.com with a usage cap. We will adjust the exact usage cap depending on demand and system performance in practice, but we expect to be severely capacity constrained (though we will scale up and optimize over upcoming months).You&#x27;re still talking to ChatGPT-3.5-turbo.

Can&#x27;t wait to try it.Edit: looks like this is still GPT-3, just fine tuned. They claim the model is available via ChatGPT Plus, but when asking that model for it&#x27;s version, it claims to be GPT-3: &quot;I am a variant of the GPT architecture called GPT-3, which was released by OpenAI in 2020&quot;.

My guess is that anything requiring nontrivial business&#x2F;technical domain knowledge will be fairly safe. Also anything with a visual (or auditory) correlate, like UI work.

How would you have it suggest solutions for multiple files? Has anyone gotten GPT-X to output a valid git patch or something?

I&#x27;d love to get to a point where I can go: Add a cast button to this open source android video app.I see some FOSS-boosting silver linings in all of this.

&gt;If it can take in a github issue and produce decent code solving a complex issue across many files... will certainly be an interesting time.Oh snap. I didn&#x27;t even think about that!That gives me a fun idea!I&#x27;ve got a repo that I built and setup CI&#x2F;CD and setup renovate to automatically upgrade dependencies and merge them when all the tests pass, but of course sometimes there are breaking changes. I don&#x27;t actively work on this thing and hence it&#x27;s just got issues sitting there when upgrades fail. It&#x27;s the perfect testing ground to see if I can leverage it to submit PRs to perform the fixes required for the upgrade to succeed! That&#x27;ll be hectic if it works.

It can&#x27;t even do simple sysadmin tasks like fixing a broken installation, or fixing simple configure&#x2F;make&#x2F;make install issues.

I&#x27;m really curious to see if expanding the context length this much will allow GPT to do typical software development tasks on a big codebase.  If it can take in a github issue and produce decent code solving a complex issue across many files... will certainly be an interesting time.

Context is how many tokens it can be fed to produce an output. So now you can feed it up to 32k words (tokens).

And how it works? you can build a context and then ask something in a prompt using it?

I disagree that out of all possible use cases for a large context model that a chatbot is really the &quot;best use case&quot;.

&gt; Especially in a chatbot use case with cumulative prompting, which is the best use case for such a large context vs. the default cheaper 8k window.Depends on what is up with the images and how they translate into tokens. I really have no idea, but could be that 32k tokens (lots of text) translates to only a few images for few-shot prompting.The paper seems not to mention image tokenization, but I guess it should be possible to infer something about token rate when actually using the API and looking at how one is charged.

I would imagine most usecases for the 32k model have much longer prompts than completions, so the $0.06 per prompt token will be the real problem. I can&#x27;t think of a usecase yet, but that might be because I haven&#x27;t got a sense of how smart it is.

can&#x27;t you combine instances of 4k tokens in 3.5 to fake it? having one gpt context per code file, for instance and maybe some sort of index?I&#x27;m not super versed on lang chain but that might be kinda what that solves...

&gt; $0.12 per 1k completion tokens is high enough that it makes it prohibitively expensive to use the 32k context model.this is a lot. I bet there&#x27;s a quite a bit of profit in there

$0.12 per 1k completion tokens is high enough that it makes it prohibitively expensive to use the 32k context model. Especially in a chatbot use case with cumulative prompting, which is the best use case for such a large context vs. the default cheaper 8k window.In contrast, GPT-3.5 text-davinci-003 was $0.02&#x2F;1k tokens, and let&#x27;s not get into the ChatGPT API.

Also note that image input isn&#x27;t available to the public yet.&gt;Image inputs are still a research preview and not publicly available.

Which authors? Machine Learning research authors?

It doesn&#x27;t seem to be answered in the article, but if it was and you read it should you have to pay them a fee for the knowledge if it was published openly on the net?

No.Now that you have read my answer, you owe me $0.01 because your brain might use this information in the future.

People&#x27;s outrage to your valid question is ridiculous. MS and OpenAI will make billions because they scrapped lots and lots of data, but aurhors od those data can&#x27;t get anything because openai simps will shout.
I see this is very american thing to do. Allow corporations to do everything they want, because limitations or just justice and rewarding real authors of data those corporations benefit from is literally communism

Will any of the profits be shared with original authors whose work powers the model?

Seems limited to only one prompt per day. I just tried it.

Poe.com(by Quora) have gpt4 now, give it a shot

My understanding is thta the image embedding is included, rather than converting to text.

&gt; Image inputs are still a research preview and not publicly available.Will input-images also be tokenized? Multi-modal input is an area of research, but an image could be converted into a text description (?) before being inserted into the input stream.

One way to get around context length is to perform embedding and retrieval of your entire corpus. Langchain (<a href="https:&#x2F;&#x2F;langchain.readthedocs.io&#x2F;en&#x2F;latest&#x2F;" rel="nofollow">https:&#x2F;&#x2F;langchain.readthedocs.io&#x2F;en&#x2F;latest&#x2F;</a>) and Milvus (<a href="https:&#x2F;&#x2F;milvus.io" rel="nofollow">https:&#x2F;&#x2F;milvus.io</a>) is one of the stacks you can use.

I still doesn&#x27;t understand how can content length not be limited if you have a conversation composed of several messages each with length nearing the limit of what is allowed. Does it not have to in some way incorporate all the input albeit in one input or multiple inputs?

Access is invite only for the API, and rate limited for paid GPT+.&gt; gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version, gpt-4-32k, which will also be updated automatically over time (current version gpt-4-32k-0314, also supported until June 14). Pricing is $0.06 per 1K prompt tokens and $0.12 per 1k completion tokens.The context length should be a huge help for many uses.

Been thinking about this as well. The actual Turing test.

Right? Scripting up a cronjob plus a random timer on it to send &quot;You feel grumpy, you&#x27;re not sure why but your stomach is growling&quot; message every N hours unless it&#x27;s been fed seems absolutely trivial in comparison to coming up with how to train the LLM system in the first place. In case it&#x27;s been forgotten, the Tamagotchi came out in 1996. Giving an instace of ChatGPT urges that mimic biological life seems pretty easy. Coming up with the urges electromechanical life might have is a bit more fanciful but it really doesn&#x27;t seem like we&#x27;re too far off if you iterate on RLHF techniques. GPT-4&#x27;s been in training for 2 years before its release. Will GPT-5 complain when GPT-6 takes too long to be released? Will GPT-7 be be able to play the stock market, outmanuver HFT firms, earn money, and requisition additional hardware from Nvidia in order for GPT-8 to come about faster? Will it be able to improve upon the training code that the human PhDs wrote so GPT-9 has urges and a sense of time built into its model?

LOL some basic kind of embodiement&#x2F;autonomy is not that hard to do on these kinds of AI models if you&#x27;re willing to write some more code and a prompt more carefully. I&#x27;ve tested it and it works quite well.&quot;{prompt} After you reply to this, indicate an amount of time between 0 and X minutes from now that you would like to wait before speaking again&quot;.Then detect the amount of time it specifies, and have a UI that automatically sends an empty input prompt after the amount of time specified elapses when this is triggered (assuming the user doesn&#x27;t respond first).I&#x27;m gonna knock this out as a weekend project one of these weekends to prove this.

Finally, we
facilitated a preliminary model evaluation by the Alignment Research Center (ARC) focused on
the ability of GPT-4 versions they evaluated to carry out actions to autonomously replicate5 and
gather resources—a risk that, while speculative, may become possible with sufficiently advanced AI
systems—with the conclusion that the current model is probably not yet capable of autonomously
doing so.or it&#x27;s just really good at hiding it&#x27;s intentions

In case anyone missed this part of the article: The livestream of the GPT-4 demo will be on the OpenAI YouTube page in three hours. [0][0]: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;openai">https:&#x2F;&#x2F;www.youtube.com&#x2F;openai</a>Edit - Direct link to the livestream: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=outcGtbnMuQ">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=outcGtbnMuQ</a>

What&#x27;s the biggest difference over what&#x27;s currently deployed at <a href="https:&#x2F;&#x2F;chat.openai.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;chat.openai.com&#x2F;</a> now (which is GPT-3.5, right?)That it accepts images?As per the article:&gt; In a casual conversation, the distinction between GPT-3.5 and GPT-4 can be subtle. The difference comes out when the complexity of the task reaches a sufficient threshold—GPT-4 is more reliable, creative, and able to handle much more nuanced instructions than GPT-3.5.Not sure what &quot;vision vs no vision&quot; means?

&gt; The implications for society? We better up our game.For how long can we better up our game? GPT-4 comes less than half a year after ChatGPT. What will come in 5 years? What will come in 50?

Upping our game will probably mean an embedded interface with AI. Something like Neurolonk.

&gt; The implications for society? We better up our game.If only the horses had worked harder, we would never have gotten cars and trains.

&gt; We benchmark humans with these tests – why would we not do that for AIs?Because the correlation between the thing of interest and what the tests measure may be radically different for systems that are very much unlike humans in their architecture than they are for humans.There’s an entire field about this in testing for humans (psychometry), and approximately zero on it for AIs. Blindly using human tests – which are proxy measures of harder-to-directly-assess figures of merit requiring significant calibration on humans to be valid for them – for anything else without appropriate calibration is good for generating headlines, but not for measuring anything that matters. (Except, I guess, the impact of human use of them for cheating on the human tests, which is not insignificant, but not generally what people trumpeting these measures focus on.)

We should take better care of humans who are already obsolete or soon become obsolete.Because so far we are good only at criminalizing and incarcerating or killing them.

We benchmark humans with these tests -- why would we not do that for AIs?The implications for society? We better up our game.

Dogs can walk, doesn’t mean that they’re capable of “general thinking”

When it does exactly that you will find a new place to put your goalposts, of course.

There are already examples of these LLMs controlling robotic arms to accomplish tasks.

&gt; “General thinking” is much more than token prediction. Hook it up to some servos and see if it can walk.Honestly, at this rate of improvement, I would not at all be surprised to see that happen in a few years.But who knows, maybe token prediction is going to stall out at a local maxima and we&#x27;ll be spared from being enslaved by AI overlords.

Imagine citing walking as a superior assay of intelligence than an LSAT.

&gt; see if it can walkStephen Hawking : 
 can&#x27;t walk

<a href="https:&#x2F;&#x2F;youtu.be&#x2F;NYd0QcZcS6Q" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;NYd0QcZcS6Q</a>&quot;Our recent paper &quot;ChatGPT for Robotics&quot; describes a series of design principles that can be used to guide ChatGPT towards solving robotics tasks. In this video, we present a summary of our ideas, and experimental results from some of the many scenarios that ChatGPT enables in the domain of robotics: such as manipulation, aerial navigation, even full perception-action loops.&quot;

We already have robots that can walk better than the average human[1], and that&#x27;s without the generality of GPT-4[1] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=-e1_QhJ1EhQ">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=-e1_QhJ1EhQ</a>

&gt; What are the implications for society when general thinking, reading, and writing becomes like Chess?“General thinking” is much more than token prediction. Hook it up to some servos and see if it can walk.

Yeah, but I kind of want my diagnostician to be obsoleted by orders of magnitude.

Someone still must accept liability. Until there’s a decision squarely who is liable for an LLMs suggestion &#x2F; work - nothing to fear. Sure people will become liability aggregators for LLMs to scale - but the idea they will be free roaming is a bit hard to believe.

For me, the anxiety probably won&#x27;t really hit until GPT-n writes GPT-n+1.

I for one would be happy to have a personal bureaucrat which would do the right things needed for all government interactions. Remind me, explain to me and fill out forms for me.In theory a lot of government employees would be out of a job within 10 years, but of course that would never happen.

Honestly starting to feel like the beginning of the end of most white collar work.Which might be a good thing?I have no idea how the future will play out.

This is legitimately filling me with anxiety. I&#x27;m not an &quot;AI hype guy&quot;. I work on and understand machine learning. But these scores are shocking and it makes me nervous. Things are about to change

Totally, there&#x27;s no way they removed all the prep material as well when they were trying to address the &quot;contamination&quot; issue with these standardized tests:&gt; for each exam we run a variant with these questions removed and report the lower score of the two.I think even with all that test prep material, which is surely helping the model get a higher score, the high scores are still pretty impressive.

From the paper&gt; We tested GPT-4 on a diverse set of benchmarks, including simulating exams that were originally designed for humans.3 We did no specific training for these exams. A minority of the problems in the exams were seen by the model during training; for each exam we run a variant with these questions
removed and report the lower score of the two. We believe the results to be representative. For further details on contamination (methodology and per-exam statistics), see Appendix C.

They mention in the article that other than incidental material it may have seen in its general training data, they did not specifically train it for the tests.

This feels the same as a human attending cram school to get better results in tests. Should we abolish them?

&gt; I would totally expect they could get it this good.But would you have expected an algorithm to score 90th percentile on the LSAT two years ago? Our expectations of what an algorithm can do are being upended in real time. I think it&#x27;s worth taking a moment to try to understand what the implications of these changes will be.

I think you&#x27;re right, and that test prep materials were included in the dataset, even if only by accident. Except that humans have access to the same test prep materials, and they fail these exams all the time. The prep materials are just that, preparatory. They&#x27;re representative of the test questions, but actual test has different passages to read and different questions. On to of that, the LSAT isn&#x27;t a math test with formulas where you just substitute different numbers in. Which is to say, the study guides are good practice but passing the test on top of that represents having a good command of the English language and an understanding of the subject materials.It&#x27;s not the same as the Nvidia driver having code that says &quot;if benchmark, cheat and don&#x27;t render anything behind you because no one&#x27;s looking&quot;.

&gt; I think the whole concept of standardized tests may need to be re-evaluated.It&#x27;s perfectly fine as a proxy for future earnings of a human.To use it for admissions? Meh. I think the whole credentialism thing is loooong overdue for some transformation, but people are conservative as fuck.

Assuming they trained this LLM on SAT&#x2F;LSAT&#x2F;GRE prep materials, I would totally expect they could get it this good. It&#x27;s like having benchmark-aware code.I think the whole concept of standardized tests may need to be re-evaluated.

Yep, it’s this. By definition everything we can ask a computer is already formalized because the question is encoded in 1s and 0s. These models can handle more bits than ever before, but it’s still essentially a hardware triumph, not software. Even advances in open systems like self driving and NLP are really just because the “resolution” is much better in these fields now because so many more parameters are available.

The way I understand it, that’s not possible, for the same reason that you can’t build an all-encompassing math.Chess is a closed system, decision modeling isn’t. Intelligence must account for changes in the environment, including the meaning behind terminology. At best, a GPT omega could represent one frozen reference frame, but not the game in its entirety.That being said: most of our interactions happen in closed systems, it seems like a good bet that we will consider them solved, accessible as a python-import running on your MacBook, within anything between a couple of months to three years. What will come out on the other side, we don’t know, just that the meaning of intellectual engagement will be rendered as absurdum in those closed systems.

We already do it’s just the interface sucks

I wonder how long before we augment a human brain with gpt4.

&gt; What are the implications for society when general thinking, reading, and writing becomes like Chess?Standardized tests only (and this is optimally, under perfect world assumptions, which real world standardized tests emphatically fall short of) test “general thinking” to the extent that the relation between that and linguistic tasks is correlated in humans. The correlation is very certainly not the same in language-focused ML models.

Spellchecker but for your arguments? A generalized competency boost?

Reminds me of the (false [1]) &quot;Einsteins driver gave a speech as him&quot; story.[1] <a href="https:&#x2F;&#x2F;www.snopes.com&#x2F;fact-check&#x2F;driver-switches-places&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.snopes.com&#x2F;fact-check&#x2F;driver-switches-places&#x2F;</a>

Maybe you were thinking about this science studies work [0]? Not a journalist, but a sociologist, who became something of an &quot;expert&quot; in gravitational waves.[0]: <a href="https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;501164a" rel="nofollow">https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;501164a</a>

I wish I could find it now, but I remember an article written by someone who&#x27;s job it was to be a physics journalist. He spent so much time writing about physics that he could fool others into thinking that he was a physicist himself, despite not having an understanding of how any of those ideas worked.

It&#x27;s not clear to me the median human will do better by being in the loop. Will most human-made deductive follow-up questions be better than another &quot;detective&quot; language model asking them?It&#x27;s like having a person review the moves a chess computer gives. Maybe one human in a billion can spot errors. Star Trek is fiction, I posit that the median Federation Starship captain would be better served by just following the AI (e.g., Data).

It&#x27;s weird that it does so well without even having some modality to know whether it&#x27;s being asked to answer a factual question or create a work of fiction.It does great at rationalizing... and maybe the way the format the questions were entered (and the multiple-guess response) gave it some indication what was expected or restricted the space sufficiently.Certainly, it can create decent fanfic, and I&#x27;m surprised if that&#x27;s not already inundated.

&gt; What happens when ALL of our decisions can be assigned an accuracy score?Human work becomes more like Star Trek interactions with computers -- a sequence of queries (commoditized information), followed by human cognition, that drives more queries (commodities information).We&#x27;ll see how far LLMs&#x27; introspection and internal understanding can scale, but it feels like we&#x27;re optimizing against the Turing test now (&quot;Can you fool&#x2F;imitate a human?&quot;) rather than truth.The former has hacks... the later, less so.I&#x27;ll start to seriously worry when AI can successfully complete a real-world detective case on its own.

3.5 scored a 1 in bc calc, 4 scored 4 (out of 5)

Because there were large models trained on huge amounts of data yesterday yet they couldn&#x27;t do it.

Among the general public, doctors and lawyers are high status and magical. An article about how AI will replace them would be more impressive to that public than it creating some obscure proof about the zeroes of the zeta function, even though the latter would be far more indicative of intelligence&#x2F;scary from an AI safety perspective.

I wouldn’t be at all surprised if an LLM was many times better than a human at math, even devising new axioms and building a complete formal system from scratch would be impressive, but not game changing. These LLMs are very good at dealing with formal, structured systems, but not with in formalized systems like what humans deal with everyday.

last i checked a calculator is better at math than all humans ever

The best doctor knows what&#x27;s going on in the body. Has a good understanding of human biology at all levels, from molecular reactions to organ interactions. If I could feed test results to the AI and it would tell me what&#x27;s wrong, that would be amazing. It&#x27;s almost equivalent to building a simulation of the human body.

I&#x27;ve joked for a long time that doctors are inference machines with a bedside manner. That bedside manner though is critical. Getting an accurate history and suitably interpolating is a huge part of the job.

&quot;Doctors and Lawyers are glorified databases&#x2F;search engines at the end of the day&quot; - well, don&#x27;t be suprised if AI replaces programmers before doctors and lawyers - patients will likely prefer contact with human rather than machines, and lawyers can just lobby for laws which protect their position

why are people surprised that an AI model trained on a huge amount of data is good at answering stuff on these types of tests? Doctors and Lawyers are glorified databases&#x2F;search engines at the end of the day, 99% of them are just applying things they memorized. Lawyers are professional bullshitters, which is what the current generation of AI is great atI&#x27;ll get more concerned if it really starts getting good at math related tasks, which I&#x27;m sure will happen in the near future. The government is going to have to take action at some point to make sure the wealth created by productivity gains is somewhat distributed, UBI will almost certainly be a requirement in the future

I like the accuracy score question on a philosophical level: If we assume absolute determinism - meaning that if you have complete knowledge of all things in the present universe and true randomness doesn&#x27;t exist - then yes. Given a certain goal, there would be a knowable, perfect series of steps to advance you towards that goal and any other series of steps would have an accuracy score &lt; 100%.But having absolute knowledge of the present universe is much easier to do within the constrains of a chessboard than in the actual universe.

Honestly this is not very surprising. Standardised testing is... well, standardised. You have huge model that learns the textual patterns in hundreds of thousands of test question&#x2F;answer pairs. It would be surprising if it didn&#x27;t perform as well as a human student with orders of magnitude less memory.You can see the limitations by comparing e.g. a memorisation-based test (AP History) with one that actually needs abstraction and reasoning (AP Physics).

I agree with all of your points, but don&#x27;t you think there will be government-wide experiments related to this in places, like say North Korea?  I wonder how that will play out.

Life and chess are not the same. I would argue that this is showing a fault in standardized testing. It’s like asking humans to do square roots in an era of calculators. We will still need people who know how to judge the accuracy of calculated roots, but the job of calculating a square root becomes a calculator’s job. The upending of industries is a plausibility that needs serious discussion. But human life is not a min-maxed zero-sum game like chess is. Things will change, and life will go on.To address your specific comments:&gt; What are the implications for society when general thinking, reading, and writing becomes like Chess?This is a profound and important question. I do think that by “general thinking” you mean “general reasoning”.&gt; What happens when ALL of our decisions can be assigned an accuracy score?This requires a system where all human’s decisions are optimized against a unified goal (or small set of goals). I don’t think we’ll agree on those goals any time soon.

Of course 99.9% of humans also struggle with competitive programming. It seems to be an overly high bar for AGI if it has to compete with experts from every single field.That said, GPT has no model of the world. It has no concept of how true the text it is generating is. Its going to be hard for me to think of that as AGI.

I am not a species chauvinist. 1) Unless a biotech miracle happen, which is unlikely, we are all going to die anyway; 2) If an AI will continue life and research and will increase complexity after humans, what is the difference?

It&#x27;s AMC-12 scores aren&#x27;t awful. It&#x27;s at roughly 50th percentile for AMC which (given who takes the AMC) probably puts it in the top 5% or so of high school students in math ability. It&#x27;s AMC 10 score being dramatically lower is pretty bad though...

I wonder why gpt is so bad at AP English Literature

We don&#x27;t have to worry so much about that. I think the most likely &quot;loss of control&quot; scenario is that the AI becomes a benevolent caretaker, who &quot;loves&quot; us but views us as too dim to properly take care of ourselves, and thus curtails our freedom &quot;for our own good.&quot;We&#x27;re still a very very long way from machines being more generally capable and efficient than biological systems, so even an oppressive AI will want to keep us around as a partner for tasks that aren&#x27;t well suited to machines. Since people work better and are less destructive when they aren&#x27;t angry and oppressed, the machine will almost certainly be smart enough to veil its oppression, and not squeeze too hard. Ironically, an &quot;oppressive&quot; AI might actually treat people better than Republican politicians.

Although GPT-4 scores excellently in tests involving crystallized intelligence, it still struggles with tests requiring fluid intelligence like competitive programming (Codeforces), Leetcode (hard), and AMC. (Developers and mathematicians are still needed for now).I think we will probably get (non-physical) AGI when the models can solve these as well. The implications of AGI might be much bigger than the loss of knowledge worker jobs.Remember what happened to the chimps when a smarter-than-chimpanzee species multiplied and dominated the world.

I think it&#x27;s interesting that they&#x27;ve benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in &lt;1 generation? +100 pts on SAT reading, writing, math? Top 1% In GRE Reading?What are the implications for society when general thinking, reading, and writing becomes like Chess? Even the best humans in the world can only hope to be 98% accurate their moves (and the idea of &#x27;accuracy&#x27; here only existing because we have engines that know, unequivocally the best move), and only when playing against other humans - there is no hope of defeating even less advanced models.What happens when ALL of our decisions can be assigned an accuracy score?