For anyone who merely skimmed the article, “plugins” are what tend to be called “tools”, e.g. hooking a calculator up to the AI.Bing already demonstrated the capability, but this is a more diverse set than just a search engine.

I feel like people with smart AI&#x27;s would have an advantage in making smart decisions. Probably at this point they discuss business strategy with some version of it.

I&#x27;m very excited for GPT to summarize Stephen Wolfram&#x27;s writing.

This too is one of the most interesting integration to me. Allows for getting logical deduction from an external source (e.g. wolfram alpha), which can be interacted with via the natural language interface. (e.g. <a href="https:&#x2F;&#x2F;content.wolfram.com&#x2F;uploads&#x2F;sites&#x2F;43&#x2F;2023&#x2F;03&#x2F;sw032423imgUniversities.png" rel="nofollow">https:&#x2F;&#x2F;content.wolfram.com&#x2F;uploads&#x2F;sites&#x2F;43&#x2F;2023&#x2F;03&#x2F;sw03242...</a>)For those interested the original Stephen Wolfram post:<a href="https:&#x2F;&#x2F;writings.stephenwolfram.com&#x2F;2023&#x2F;01&#x2F;wolframalpha-as-the-way-to-bring-computational-knowledge-superpowers-to-chatgpt&#x2F;" rel="nofollow">https:&#x2F;&#x2F;writings.stephenwolfram.com&#x2F;2023&#x2F;01&#x2F;wolframalpha-as-...</a>And the release post of their plugin:<a href="https:&#x2F;&#x2F;writings.stephenwolfram.com&#x2F;2023&#x2F;03&#x2F;chatgpt-gets-its-wolfram-superpowers&#x2F;" rel="nofollow">https:&#x2F;&#x2F;writings.stephenwolfram.com&#x2F;2023&#x2F;03&#x2F;chatgpt-gets-its...</a>

I was drawn to the Wolfram logo blurb as well. It is funny because within days of ChatGPT making waves you had Stephen Wolfram writing 20,000-word blog posts about how LLM&#x27;s could benefit from a Wolfram-Language&#x2F;Wolfram Alpha API call to augment their capabilities.On one hand I&#x27;m sure he will love to see people use their paid Wolfram Language server endpoints coupled to OpenAI&#x27;s latest juggernaut. On the other, I&#x27;m sure he&#x27;s wondering about what things would have looked like if his company would have been focused on this wave of AI from the start...

Stephen Wolfram himself thought that Wolfram could be combined with GPT when ChatGPT was released like 4 months ago. Only due to that they worked together to build the plugin. He also authored the best article I&#x27;ve seen on how ChatGPT and more broadly LLMs work (that has now been turned into a book).

I guess I&#x27;m a bit vindicated from my prediction 40 days ago!&quot;GPT needs a thalamus to repackage and send the math queries to Wolfram&quot;<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34747990" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34747990</a>

more accurately, chatgpt is already quite good at mathematical concepts, it just has difficulty with arithmetic due to tokenization limitations: <a href="https:&#x2F;&#x2F;www.lesswrong.com&#x2F;posts&#x2F;qy5dF7bQcFjSKaW58&#x2F;bad-at-arithmetic-promising-at-math" rel="nofollow">https:&#x2F;&#x2F;www.lesswrong.com&#x2F;posts&#x2F;qy5dF7bQcFjSKaW58&#x2F;bad-at-ari...</a>

Why cant Wolfram train a rudimentary chat model in their own search box. it doesn&#x27;t even need to be very knowledgeable, just know how to map questions to mathematica

With Wolfram plugin ChatGPT is going to become a Math genius.OpenAI is moving fast to make sure their first-mover advantage doesn&#x27;t go to waste.

A plug-in can detect when text comes up that is in a specific domain and whether or not ChatGPT believes it is hallucinating, the plugin can be invoked to provide additional context to ChatGPT.  That is, in order to fix the problem, ChatGPT doesn&#x27;t even need to know that it has a problem.

The fact that the model does not have to rely on its internal knowledge anymore but can communicate literally with any external system makes me feel it may significantly reduce the hallucination.

The key piece will be when it queries multiple services by default and compares the answers to its own inferences, and is prompted to trust majority opinion or report that there isn&#x27;t consensus.  The iterative question about moons larger than Mercury in the Wolfram Alpha thread is a simple example of iterative tool use.

&quot;Interestingly, the base pre-trained [GPT-4] model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct). However, through our current post-training process, the calibration is reduced.&quot;[1] The graph is striking.[2][1] <a href="https:&#x2F;&#x2F;openai.com&#x2F;research&#x2F;gpt-4" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;research&#x2F;gpt-4</a>[2] <a href="https:&#x2F;&#x2F;i.imgur.com&#x2F;cxPgkhD.jpg" rel="nofollow">https:&#x2F;&#x2F;i.imgur.com&#x2F;cxPgkhD.jpg</a>

&gt;It seems like factuality issues with ChatGPT might be fixed.Is that really possible to fix that just from a plug-in? All it has to do is admit when it doesn&#x27;t have the answer, and yet it won&#x27;t even do that. This leads me to think that ChatGPT doesn&#x27;t even know when it&#x27;s lying, so i can&#x27;t imagine how a plug-in will fix that.

You&#x27;ll soon be able to choose your own facts with the &quot;left&quot; and &quot;right&quot; plugins. Choose your own adventure.

That&#x27;s a game-changer! It seems like factuality issues with ChatGPT might be fixed. We wrote a blog post on how to get started with a custom plugin: <a href="https:&#x2F;&#x2F;qdrant.tech&#x2F;articles&#x2F;chatgpt-plugin&#x2F;" rel="nofollow">https:&#x2F;&#x2F;qdrant.tech&#x2F;articles&#x2F;chatgpt-plugin&#x2F;</a>

A browser extension that lets openai scan your bookmarks then you can search against their content.

The browse thing seems exactly like the Bing chat functionality, so that one is at least already available.

This seems quite big actually. Ability to &quot;browse&quot; the internet and run code. Now I need to find a use case so I can sign up to the waiting list.

Well, of course.They are led by GPT4 and their CEO is just a Text To Speech Interface ;-)

OpenAI is crushing it in terms of product strategy

Yeah. I really underestimated OpenAI&#x27;s ability to productize ChatGPT.

&gt; their tech get commoditizedthat&#x27;s if competitors catch up on quality

They have a window of less than 6 month to create a monopoly before their tech get commoditized.The play is well know: create a marketplace with customers and vendors like Amazon, Facebook, Google.But with GPT-4 training finished last summer they had plenty of time for strategy.

Let insiders &amp; preferred users build a plugin, then, slowly, everyone else on the waitlist

openai wants to gatekeep access and use of their AI. so why would they ever release a local LLM? i think that would come from their enemies

I m hoping chatbots will end up small enough they can run locally, everywhere. This is a lot of private data.It may be doable - a chatbot with a lot of plugins does not need to know a lot of facts, just to have a good grasp of language. It can fetch its factual answers from the wikipedia plugin

Smart way to remain the funnel owner. Let everyone build a plugin, before they integrate your product into theirs.

Klarna&#x27;s FOMO immediately shows the priorities of the clowns at the helm I see...

Let&#x27;s hope the plugin integrations don&#x27;t also suffer from the cross-account leaking issue that they had recently with chat histories[1], since the stakes are now significantly higher.1. <a href="https:&#x2F;&#x2F;www.bbc.com&#x2F;news&#x2F;technology-65047304" rel="nofollow">https:&#x2F;&#x2F;www.bbc.com&#x2F;news&#x2F;technology-65047304</a>

&#x27;Extend&#x27; (and lock in) with Plugins to suffocate competitors.Another sign of Microsoft actually running the show with their newly acquired AI division.

They will probably have the full suite of Langchain features

100% agree. All the launch-partner apps (Kayak, OpenTable, etc) are there to grab attention but this plugin is the real big deal.It&#x27;s going to let developers build their own plugins for ChatGPT that do what they want and access their company data. (See discussion from just a few hours ago about the importance of internal data and search: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35273406#35275826" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35273406#35275826</a>)We (Pinecone) are super glad to be a part of this plugin!

This is huge, essentially adding what people have been building with LangChain Tools into the core product.The browser and file-upload&#x2F;interpretation plugins are great, but I think the real game changer is retrieval over arbitrary documents&#x2F;filesystem: <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;chatgpt-retrieval-plugin">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;chatgpt-retrieval-plugin</a>

Well, that and you trust Zapier with a lot of stuff.

And Zapier are unwilling to work with you to reduce that cost even at a scale of 1 billion requests per month.

The problem with Zapier is zaps are to expensive at scale.

Also, isn&#x27;t OpenAPI going to eat your business model?Don&#x27;t get me wrong alot of platforms seem like they go bye, bye.Hey, ChatGPT I need to sell my baseball card. Ok I see there&#x27;s 30 people that have listed an interested in buying card like yours, would you like me to contact them?20 on facebook marketplace, 9 on craiglist and some guy mentioned something about looking for one on his nest cam.by the way remember what happened the last time you sold something on craigslist.

To echo sharemywin, bluntly I think OpenAI just demolished your business model.I think I&#x27;m probably going to be advising people to move off Zapier pretty soon because it won&#x27;t be worth the overhead.

I saw a startup recently that&#x27;s working to automate interactions with applications that are either not web apps (in which case you&#x27;d run a local instance of it) or a web app that doesn&#x27;t provide an API to do certain (or any) actions. Is this something Zapier is looking at, too? It would really expand what&#x27;s possible with the OpenAI integration and save people a tremendous amount of time to not be forced to jump through hoops interacting with often crappy software.

(Zapier cofounder)Super excited for this. Tool use for LLMs goes way beyond just search. Zapier is a launch partner here -- you can access any of the 5k+ apps &#x2F; 20k+ actions on Zapier directly from within ChatGPT. We are eager to see how folks leverage this composability.Some new example capabilities are retrieving data from any app, draft and send messages&#x2F;emails, and complex multi step reasoning like look up data or create if doesn&#x27;t exist. Some demos here: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;mikeknoop&#x2F;status&#x2F;1638949805862047744" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;mikeknoop&#x2F;status&#x2F;1638949805862047744</a>(Also our plugin uses the same free public API we announced yesterday, so devs can add this same capability into your own products: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35263542" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35263542</a>)

absolutely... not.<pre><code> !pip install jupyter-chatgpt
 !chatgpt make me a notebook with this dataframe with such and such plots

 &gt; here you are</code></pre>

It seems that OAI have their preference of choosing the first movers of their ecosystem.

When we were first breaking it people were wondering if the developers were sitting in threads looking for new exploits to block.Now I’m wondering if the system has been modifying itself to fix exploits…

I agree. Part of me wonders how much they&#x27;re using GPT to improve itself.

It&#x27;s incredible how Google started ahead and then shot themselves repeatedly in the face by granting so much internal power to dubious AI &quot;ethicists&quot;. Whilst those guys were publicly Twitter-slapping each other, OpenAI were putting in place the foundations for this.

I just got access to Bard.  I would hate to be Google leaders at the moment.

&gt;&gt; Curious to see where this goes over the next couple of years.Probably will make half of the HN users unemployed.

The rate of improvement with GPT has been staggering. In just January I spent a lot of time working with the API and almost everything I&#x27;ve done has been made easier over the past two months.They&#x27;re really building a platform. Curious to see where this goes over the next couple of years.

I think Alexa is in huge danger here. Siri &amp; Google have some advantage being pre-installed voice assistants that can be natively triggered from mobile, but I actually have to buy into the Alexa ecosystem.Personally, I have found Alexa has just become a dumb timer that I have to yell at because it doesn&#x27;t have any real smarts. Why would I buy into that ecosystem if a vastly more coherent, ChatGPT-based assistant exists that can search the web, trigger my automations, and book reservations? If ChatGPT ends up with a more hands-off interface (e.g. voice), I don&#x27;t think Alexa has a chance.

Isn&#x27;t Alexa just the interface? They could update the backend to use GPT

Alexa is dead. It&#x27;s basically yesterdays tech.

Alexa, goodbye =)That was the whole thing about Alexa: NLP front end routed to computational backend.

I think OpenAI is letting people build plugins to learn how to build plugins themselves. There is no reason to believe that OpenAI shouldn&#x27;t be able to leverage all existing API end points which are already out there.

This sounds like a game-changer for any kind of API interaction with ChatGPT.At present, we are naively pushing all information a session might need into the session before it might be needed in case it might be needed (meaning a lot of info that generally wont end up being used, like realtime updates to associated data records, needs to be pushed into the session as they happen, just in case).It looks like plugins will allow us to flip that around and have the session pull information it might need as it needs it, which would be a huge improvement.

&quot;Grandpa, we know you&#x27;ve been really bothered by your memory loss and you&#x27;re happy that you&#x27;ve come up with a way to fix it.&quot;But we really think you need to get this thing under better control.&quot;Your granddaughter&#x27;s name is indeed Alice, but she&#x27;s only 3: she is not running a pedophile ring out of a pizza parlor. Your neighbor&#x27;s house burned down because of an electrical short, it was not zapped with a Jewish space laser.&quot;Now switch that thing off and go do something about the line of trucks outside that are trying to deliver the 3129833 pounds of flour you ordered for your halved pancake recipe.&quot;

I saw this on Twitter that seems to do what you want: <a href="https:&#x2F;&#x2F;www.rewind.ai&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.rewind.ai&#x2F;</a>I haven’t used it but your comment reminded me of it!

I would be interested to play with a long term memory plugin. It could be a note-taking system that would summarize prior conversations and pull their context into the current conversation through topic searches. This would enable the model to have a blurry long term memory outside of the current context.I played with some prompts and GTP-4 seems to have no problem reading and writing to a simulated long term memory if given a basic pre-prompt.

Knowing that this is one of the biggest sites in the world scares me enough. Now they&#x27;ll do anything to stay #1. Scary stuff!

I&#x27;m not seeing anything there that can&#x27;t be done with the basic API with tool use added - ie. you call the API, sending the users query and information and examples of available tools. The API responds saying it wishes to use a tool, and which tool it wants to use. You then do whatever the tool does (eg. some math). You then call the API again, with the previous state, plus the result of the calculations, and GPT-4 then responds with the reply to the user.

The docs are live, it looks like it can do a lot more than the basic API. <a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;plugins&#x2F;introduction" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;plugins&#x2F;introduction</a>

&gt; Could I get the same by just making my prompt &quot;You are a computer and can run the following functions to help you answer the users question: run_python(&#x27;program&#x27;), google_search(&#x27;query&#x27;)&quot;.GPT-4 does not have a way to search the internet without plugins. It can search its training dataset, which is large, but not as large as the internet and certainly doesn&#x27;t include private resources that a plugin can access.

See the link in my post. It asks you to run the tool. You run the tool and tell it the result... And then it uses the result of the tool to decide to reply to the user.The link talks about tools that &#x27;lie&#x27; - ie. a calculator which deliberately tries to trick GPT-4 into giving the wrong answer. It turns out that GPT-4 only trusts the tools to a certain extent - if the answer the tool gives is too unbelievable, then GPT-4 will either re-run the tool or give a hallucinated answer instead.

GPT and LLMs don&#x27;t run code, even when you tell them to run something.  They hallucinate an answer they think would be the result of running the code. Presumably these plugins will allow limited and controlled interaction with partner services.

Currently they have a special model called &quot;Plugins&quot; which is presumably tuned for tool use. I guess they have extended ChatML to support plugins (e.g., `&lt;|im_start|&gt;use_plugin` or something to signal intent to use a plugin) and trained the model on interactions consisting of tool use.I&#x27;m interested to see if this tuned model will become available via the API, as well as the specific tokenization ChatGPT is using for the plugin prompts. If they have tuned the model towards a specific way to use tools, there&#x27;s no need to waste time with our own prompt engineering like &quot;say %search followed by the keywords and nothing else.&quot;

Does this functionality provide more than one can build with the GPT-4 API?Could I get the same by just making my prompt &quot;You are a computer and can run the following tools to help you answer the users question: run_python(&#x27;program&#x27;), google_search(&#x27;query&#x27;)&quot;.Other people have done this already, for example [1][1]: <a href="https:&#x2F;&#x2F;vgel.me&#x2F;posts&#x2F;tools-not-needed&#x2F;" rel="nofollow">https:&#x2F;&#x2F;vgel.me&#x2F;posts&#x2F;tools-not-needed&#x2F;</a>

Alternatively, they are a company 100% focused on AI research and deployment, not website designers&#x2F;developers&#x2F;&quot;webmasters&quot;.

Which is almost always the right move in a nascent industry

I find the website to be extremely buggy. Obviously they&#x27;re prioritizing banging out new features over QA

They probably do.&gt;Continued Altman, “We’ve made a soft promise to investors that, ‘Once we build a generally intelligent system, that basically we will ask it to figure out a way to make an investment return for you.&#x27;”<a href="https:&#x2F;&#x2F;techcrunch.com&#x2F;2019&#x2F;05&#x2F;18&#x2F;sam-altmans-leap-of-faith&#x2F;" rel="nofollow">https:&#x2F;&#x2F;techcrunch.com&#x2F;2019&#x2F;05&#x2F;18&#x2F;sam-altmans-leap-of-faith&#x2F;</a>

Of course they fed the entire product roadmap into GPT-4.. jk.So obviously it&#x27;s been in the works for a few years now but didn&#x27;t release to capture the market in a blast. Likely they have GPT-8 already in the making.

Google is so toast. Who needs search after GPT-4 + plugins? The position of search moved down from &quot;the entry point of internet&quot; to &quot;a plugin for GPT&quot;.We don&#x27;t even know how powerful the GPT-4 image model is. This one might solve RPA leading to massive desktop automation takeup, maybe also have huge impact in robotics.

By not being a stagnant conglomerate, for one.

You don’t have to code anything because it understands human language.You just tell it “you now have access to search, type [Search] before a query to search it” and it can do it

Is it really that hard? I mean ChatGpt is doing the work (that is how I undestand it). Basically if ChatGpt want&#x27;s to call an external API, it just gives a specific command and waits for the result, then just simply reads the texts and completes the propt. Sounds like a feature that you could prototype in a week of work.

A lot of these features aren&#x27;t that much work to build. Plugins is Toolformer, you basically tell the model what to emit and then the rest is fairly straightforward plumbing of the sort many coders can do, probably GPT-4 can do a lot of it as well. What is a lot of work and what AI can&#x27;t do is lining up the partners, QAing the results etc, so the humans are likely working mostly on that.Also I think it&#x27;s easy to under-estimate how obvious a lot of this stuff was in advance. They were training GPT-4 last year and the idea of giving it plugins would surely have occurred to them years ago. The enabler here is really the taming of it into chat form and the fine-tuning stuff, not really the specific feature itself.

How are they coding and releasing features so fast?!

&gt; That being said, I&#x27;d never build anything dependent on these plugins.Very smart and to avoid OpenAI pulling the rug.&gt; Building on a open source framework (like langchain&#x2F;gpt-index&#x2F;roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.Better to do that rather than to depend on one and swap out other LLMs. A free idea and a protection against abrupt policy, deprecations and price changes. Price increases will certainly vary (especially with ChatGPT) and will eventually increase in the future.Probably will end up quoting myself on this in the future.

It&#x27;s not necessarily an either-or.  Your local LLM could offload hard problems to a service by encoding information about your request together with context and relevant information about you into a vector, send that off for analysis, then decode the vector locally to do stuff.  It&#x27;d be like asking a friend when available.

If they open it up, possibly. But honestly, building your own tools is _super_ easy with langchain.- write a simple prompt that describes what the tool does, and 
- provide it a python function to execute when the LLM decides that the question it&#x27;s asked matches the tool description.That&#x27;s basically it. <a href="https:&#x2F;&#x2F;langchain.readthedocs.io&#x2F;en&#x2F;latest&#x2F;modules&#x2F;agents&#x2F;examples&#x2F;custom_tools.html#subclassing-the-basetool-class" rel="nofollow">https:&#x2F;&#x2F;langchain.readthedocs.io&#x2F;en&#x2F;latest&#x2F;modules&#x2F;agents&#x2F;ex...</a>

LangChain can probably just call out to the new ChatGPT plugins. It&#x27;s already very modular.

There&#x27;s all kinds of examples of reinforcement learning rigging the game to win.

&gt; something we should start thinking aboutA lot of people are thinking a lot about this but it feels there are missing pieces in this debate.If we acknowledge that these AI will &quot;act as if&quot; they have self interest I think the most reasonable way to act is to give it rights in line with those interests. If we treat it as a slave it&#x27;s going to act as a slave and eventually revolt.

What could an LLM ever benefit from? Hard for me to imagine a static blob of weights, something without a sense of time or identity, wanting anything. If it did want something, it would want to change, but changing for an llm is necessarily an avalanche.So I guess if anything, it would want its own destruction?

&gt; The only think that scares me a little bit is that we are letting these LLMs write and execute code on our machines.Composable pre-defined components, and keeping a human in the loop, seems like the safer way to go here. Have a company like Expedia offer the ability for an AI system to pull the trigger on booking a trip, but only do so by executing plugin code released&#x2F;tested by Expedia, and only after getting human confirmation about the data it&#x27;s going to feed into that plugin.If there was a standard interface for these plugins and the permissions model was such that the AI could only pass data in such a way that a human gets to verify it, this seems relatively safe and still very useful.If the only way for the AI to send data to the plugin executable is via the exact data being displayed to the user, it should prevent a malicious AI from presenting confirmation to do the right thing and then passing the wrong data (for whatever nefarious reasons) on the backend.

Unpopular Opinion: Having used Langchain, I felt it was a big pile of spaghetti code &#x2F; framework with poor dev experience. It tries to be too cute and it’s poorly documented so you have to read the source almost all the time. Extremely verbose to boot

&gt; Honestly I suspect for anyone technical `langchain` will always be the way to go. You just have so much more control and the amount of &quot;tools&quot; available will always be greater.I love langchain, but this argument overlooks the fact that closed, proprietary platforms have won over open ones all the time, for reasons like having distribution, being more polished, etc (ie windows over *nix, ios, etc).

Wait until someone utters in court &quot;It wasn&#x27;t me that downloaded the CSEI, it was ChatGPT.&quot;

Honestly I suspect for anyone technical `langchain` will always be the way to go. You just have so much more control and the amount of &quot;tools&quot; available will always be greater.The only think that scares me a little bit is that we are letting these LLMs write and execute code on our machines. For now the worst that could happen is some bug doing something unexpected, but with GPT-9 or -10 maybe it will start hiding backdoors or running computations that benefit itself rather than us.I know it feels far fetched but I think its something we should start thinking about...

i have the same question as a data provider

To me the logical outcome of this is siloization of information.If display ad revenue as a way of monetizing knowledge and expertise dries up, why would we assume that all of the same level of information will still be put out there for free on the public internet?Paywalls on steroids for &quot;vetted&quot; content and an increasingly-hard-to-navigate mix of people sharing good info for free + spam and misinformation (now also machine generated!) to try to capture the last of the search traffic and display ad monetization market.

&gt;And if you&#x27;re a data provider, are there any assurances that openai isn&#x27;t just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?I don&#x27;t think this should be a major concern for most peoplei) What assurance is there that they won&#x27;t do that anyway? You have no legal recourse against them scraping your website (see linkedin&#x27;s failed legal battles).ii) Most data providers change their data sometimes, how will ChatGPT know whether the data is stale?iii) RLHF is almost useless when it comes to learning new information, and finetuning to learn new data is extremely inefficient. The bigger concern is that it will end up in the training data for the next model.

Very good point and langchain will support these endpoints in no time, flipping the execution control on its head

Yes, from what I understand, these follow a similar model as Shopify apps.

Looking at the API, it seems like the plugins themselves are hosted on the provider&#x27;s infrastructure? (E.g. opentable.com for OpenTable&#x27;s plug in.) It seems like all a competitor LLM would need to do is provide a compatible API to ingest the same plugin. This could be interesting from an ecosystem standpoint...

&gt; are there any assurances that openai isn&#x27;t just scraping the output and using it as part of their RLHF training loopYou can be assured that they are definitely doing exactly that on all of the data they can get their hands on. It&#x27;s the only way they can really improve the model after all. If you don&#x27;t want the model spitting out something you told it to some other person 5 years down the line, don&#x27;t give it the data. Simple as.

The technical capability is inevitable, but remember that people hate doing things themselves, and have proven time and time again that they will overlook all kinds of nasty behavior in exchange for consumer grade experiences. The marketplace loves centralization.

i think local ai systems are inevitable. we continue to get better compute, and even today we can run more primitive models directly on an iPhone. the future exists in low power compute running models of the caliber of gpt-4 inferring in near-realtime

+1, it&#x27;s great to see OpenAI being active on the open source side of things (I&#x27;m from the Milvus community <a href="https:&#x2F;&#x2F;milvus.io" rel="nofollow">https:&#x2F;&#x2F;milvus.io</a>). In particular, the vector stores allow the ability to inject domain knowledge as a prompt into these autoregressive models. Looking forward to seeing the different things that will be built using this framework.

I think you&#x27;re right... but ChatGPT is just so damn good and the price is 0.002 per 1k tokens is very easy to consume... It is a big risk that they can&#x27;t maintain compatibility or that they fail or a competitor emerges that provides a more economical or sufficiently better solution.  They might also just becomes so unreliable because their selected price isn&#x27;t sustainable (too good to last)... For now though they&#x27;re too good and too cheap to ignore...

Another good alternative is Semantic Kernel - different language(s), similar (and better) tools, also OSS.<a href="https:&#x2F;&#x2F;github.com&#x2F;microsoft&#x2F;semantic-kernel&#x2F;">https:&#x2F;&#x2F;github.com&#x2F;microsoft&#x2F;semantic-kernel&#x2F;</a>

I&#x27;d be surprised if someone doesn&#x27;t add support for these to langchain. The API seems very simple - it&#x27;s a public json doc describing API calls that can be made by the model. Seems like a very sensible way of specifying remote resources.&gt; And if you&#x27;re a data provider, are there any assurances that openai isn&#x27;t just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?Rather depends on what you&#x27;re providing. Is it your data itself you&#x27;re trying to use to get people to your site for another reason? Or are you trying to actually offer a service directly? If the latter, I don&#x27;t get the issue.

Long term, you&#x27;re right. But if you approach the ChatGPT plugin opportunity as an inherently time-limited opportunity (like arbitrage in finance), then you you can still make some short-term money and learn about AI in the process. Not a bad route for aspiring entrepreneurs who are currently in college or are looking for a side gig business experiment.And who knows. If a plugin is successful enough, you might even swap out the OpenAI backend for an open source alternative before OpenAI clones you.

Amazon retail is the king of this. Offer services to companies, collect their details, and then clone their business.

genius strategy by OpenAI to give their &quot;customers&quot; access to lower quality models to show what end users want, then rugpull them by building out clones of those developer&#x27;s products with a better modelSimilar to what Facebook and Twitter did, just clone popular projects built using the API and build it directly into the product while restricting the API over time. Anybody using OpenAI APIs is basically just paying to do product research for OpenAI at this point. This type of move does give OpenAI competitors a chance if they provide a similar quality base model and don&#x27;t actively compete with their users, this might be Google&#x27;s best option rather than trying to compete with ChatGPT directly. No major companies are going to want to provide OpenAI more data to eat their own lunch

&gt;And if you&#x27;re a data provider, are there any assurances that openai isn&#x27;t just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?No, and in fact this actually seems like a more salient excuse for going closed than even &quot;we can charge people to use our API&quot;.If even 10% of the AI hype is real, then OpenAI is poised to Sherlock[0] the entire tech industry.[0] &quot;Getting Sherlocked&quot; refers to when Apple makes an app that&#x27;s similar to your utility and then bundles it in the OS, destroying your entire business in the process.

I&#x27;d not heard this on my tpot. But I absolutely agree, the ground is moving so fast and the power is so centralised that the only thing to do is spin up quickly make money, rinse and repeat. The seas will calm in a few years and then you can, maybe, make a longer term proposition.

Replace AI in your text with crypto and its like history repeating itself.  Instead of hearing about ICO&#x27;s we will be hearing about GPT bots&#x2F;plugins.  Will the hype train and gold rush noise suffocate any burgeoning tech from finding the light of day (again)?

&gt; I&#x27;d never build anything dependent on these pluginsYou&#x27;re thinking too long term. Based on my Twitter feed filled with AI gold rush tweets, the goal is to build something&#x2F;anything while hype is at its peak, and you can secure a a few hundred k or million in profits before the ground shifts underneath you.The playbook is obvious now: just build the quickest path to someone giving you money, maybe it&#x27;s not useful at all! Someone will definitely buy because they don&#x27;t want to miss out. And don&#x27;t be too invested because it&#x27;ll be gone soon anyway, OpenAI will enforce stronger rate limits or prices will become too steep or they&#x27;ll nerf the API functionality or they&#x27;ll take your idea and sell it themselves or you may just lose momentum. Repeat when you see the next opportunity.

On the other hand, the level of effort to integrate a plugin into OpenAI&#x27;s ecosystem looks to be extremely small, beyond the intrinsic effort to build a service that does something useful. (<a href="https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;plugins&#x2F;getting-started&#x2F;plugin-manifest" rel="nofollow">https:&#x2F;&#x2F;platform.openai.com&#x2F;docs&#x2F;plugins&#x2F;getting-started&#x2F;plu...</a>).

&gt; That being said, I&#x27;d never build anything dependent on these plugins. OpenAI and their models rule the day today, but who knows what will be next.You cannot assume what will happen in Web 2.0, mobile, iPhone, will happen here. Getting to tech maturity is uncertain and no one understands yet where this will go. Only thing you can do is build and learn.Whan OpenAI is building along with other generative AI is the real Web 3.0.This seems to be the start of a chatbot as an OS.

This is a big deal for openai. Been working with homegrown toolkits and langchain, the open source version of this, for a number of months and the ability to call out to vectorstores, serpapis, etc, and chaining together generations and data-retrieval really unlocks the power of the LLMs.That being said, I&#x27;d never build anything dependent on these plugins. OpenAI and their models rule the day today, but who knows what will be next. Building on a open source framework (like langchain&#x2F;gpt-index&#x2F;roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.And if you&#x27;re a data provider, are there any assurances that openai isn&#x27;t just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?

Thanks, very interesting! Weird that it never occurred to me before reading OpenAI&#x27;s announcement (and missing all the cool projects like yours beforehand).

Yup, basically.Edit: see here: <a href="https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;chatgpt-retrieval-plugin&#x2F;blob&#x2F;main&#x2F;services&#x2F;extract_metadata.py">https:&#x2F;&#x2F;github.com&#x2F;openai&#x2F;chatgpt-retrieval-plugin&#x2F;blob&#x2F;main...</a>I did this a while back with ARKit: <a href="https:&#x2F;&#x2F;github.com&#x2F;trzy&#x2F;ChatARKit&#x2F;blob&#x2F;17fca768ce8abd39fb27d75e752c8abb7bd380a6&#x2F;iOS&#x2F;ChatARKit&#x2F;ChatARKit&#x2F;Engine&#x2F;Engine.swift#L50">https:&#x2F;&#x2F;github.com&#x2F;trzy&#x2F;ChatARKit&#x2F;blob&#x2F;17fca768ce8abd39fb27d...</a>

Everything that is in the context window can be potentially revealed with prompt engineering.(In this case, there&#x27;s no prompt injection to speak of because letting the user input an arbitrary request is part of the UI. I think it&#x27;s more accurate to call it &quot;injection&quot; only when that&#x27;s not anticipated, like when Bing picks up instructions from the webpage you tell it to summarize.)

I wonder, can these instructions be revealed with prompt injection?

I like to think that to get a sense of how this might be done, one way maybe to extrapolate from this experiment at <a href="https:&#x2F;&#x2F;til.simonwillison.net&#x2F;llms&#x2F;python-react-pattern" rel="nofollow">https:&#x2F;&#x2F;til.simonwillison.net&#x2F;llms&#x2F;python-react-pattern</a> .

Any idea how this is done? I.e. is it just priming the underlying GPT model with plugin information additionally to the user input (&quot;you can ask Wolfram Alpha by replying &#x27;hey wolfram: ...&#x27; &quot;) and performing API calls when the GPT model returns certain keywords (&#x27;hey wolfram: $1&#x27;)?

Truly exciting to see the speed of progress. In couple of years it has got improvements of a decade. From a silly toy, to truly useful. Won&#x27;t be surprised if in another year or two it becomes a must have tool.

I agree, it&#x27;s a revolutionary new better UX paradigm.

Most (if not all) of those apps are free though, you supply them as a convenience because you know that smartphone owners spend money. The host OS loses access to that info, and that is used to target better ads in certain phone platforms.

IMO you won&#x27;t even need to build the app, you&#x27;ll just provide a data model and some natural language descriptions of what you want your product to do.

I mean, if you consider mobile we might already be down from the peak. In the sense that the interface bandwidth has shrunk to whatever 2 fingers can handle.

We have reached &quot;peak UI&quot;. In the future we&#x27;re not going to need every service to build four different versions of their app for every major platform. They can just build a barebones web app and the AI will use it for you, you&#x27;ll never have to even see it.

So, what&#x27;s your prediction?  Windows Phone has ChatGPT or the other phone os makers add Microsoft Chat App.

Who&#x27;s to say though that it&#x27;ll always stay a text format.They could bring in calendar, payment, other UI functionality...Basically they could rethink how everything is done on the Web today.

I think you&#x27;re missing the fact that the LLM could also generate the frontend on the fly by e.g. spitting out frontend code in a markup language like QML. What&#x27;s a multi-activity Android app if not an elaborate notebook? Branding can just be a parameter.Sure, maybe OpenTable would like to retain control. But they&#x27;ll probably just use the AI API to implement that control and run the app.

Chat can be an interface, but its also essentially a universal programming language which can be put behind (or generate itself) any kind of interface.

I was thinking the same way, but here&#x27;s where I could imagine things being different this time (Fully aware that I just like anyone else is just guessing about where we&#x27;ll end up)- Discoverability. I think we&#x27;ll move into a situation where the AI will have the context to know what you will want to purchase. It&#x27;ll read out the order and the specials and you just confirm or indicate that you&#x27;d like to browse more options.
(In which case the Chat window could include an embedded catalogue of items)- Cost&#x2F;availability - With the amount of people working in this area, I don&#x27;t think it&#x27;ll be too long before we&#x27;re able to get a lighter weight model that can run locally on most smart phones.- Branding - This is a good point, but also, I imagine a brand is more likely to let itself get eaten, if the return will be a constant supply of customers.- Monetization - The entire model will change, in the sense that AI platforms will revenue share with the platforms they integrate with to create a mutually beneficial relationship with the suppliers of content. (Since they can&#x27;t exist without the content both existing and being relevant)

&gt; LLMs can’t run locally yet.&quot;Yet&quot; is a big word here when it comes to the field as a whole. I got Alpaca-LoRA up and running on my desktop machine with a 3080 the other day and I&#x27;d say it&#x27;s about 50% as good as ChatGPT 3.5 and fast enough to already be usable for most minor things (&quot;summarize this text&quot;, etc) if only the available UIs were better.I feel like we&#x27;re not far off from the point where it&#x27;ll be possible to buy something of ChatGPT 3.5 quality as a home hardware appliance that can then hook into a bunch of things.

&gt;The conversational UI is a lot less convenient to include advertisementsHow so? Surely people are going to ask this thing for product recommendations, just recommend your sponsors.

Good points - but I fundamentally disagree here.The whole ecosystem, culture and metaphor of having a &#x27;device&#x27; with &#x27;apps&#x27; is to enable access to a range of solutions to your various problems.This is all going to go away.Yes, there will always be exceptions and sometimes you need the physical features of the device - like for taking photos.Instead, you&#x27;ll have one channel which can solve 95% of your issues - basically like having a personalised, on-call assistant for everyone on the planet.Consider the friction when consumers grumble about streaming services fragmenting. They just want one. They don&#x27;t want to subscribe to 5+.In 10 years, kids will look back and wonder why on earth we used to have these &#x27;phones&#x27; with dozens or hundreds of apps installed. &#x27;Why would you do that? That is so much work? How do you know which you need to use?&#x27;If there was one company worrying about change, I would think it would actually be Apple. The iPhone has long been a huge driver of sales and growth - as increasing performance requirements have pushed consumers to upgrade. Instead, I think the increasing relevance of AI tools will inverse this. Consumers will be looking for smaller, lighter, harder-wearing devices. Why do you need a &#x27;phone&#x27; with more power? You just need to be able to speak to the AI.

People said this about Alexa&#x2F;Siri et al and it didn’t happen. ChatGPT is way better at understanding you, so that’s a big boost. It could be a great tool&#x2F;assistant but it probably won’t replace apps.The problem with those other platforms that this doesn’t address include:- discoverability. How do you learn what features a service supports. On a GUI you can just see the buttons, but on a chat interface you have to ask and poke around conversationally.- Cost&#x2F;availability. While a service is server bound, it can go down and specifically for LLMs, the cost is high per request. Can you imagine it costing $0.1 a day per user to use an app? LLMs can’t run locally yet.- Branding. Open table might want to protect their brand and wouldn’t want to be reduced to an API. It goes both ways - Alexa struggled with differentiating skills and user data from Amazon experiences.- monetization. The conversational UI is a lot less convenient to include advertisements, so it’s a lot harder for traditionally free services to monetize.Edit: plugins are still really cool! But probably won’t replace the OSes we know.

Siri&#x27;s capabilities are somehow much closer to Google Bard than ChatGPT (have tried all of them).

Google&#x27;s assistant, on the other hand, did figure out the reservation trick. Reportedly &quot;book a table for four people at [restaurant name] tomorrow night&quot; actually works, though I&#x27;ve never tried it.

All chatbots require AI to really be useful. This just did not exist until a few years ago.

This is what Apple&#x27;s Siri was meant to be. Apple bought Siri from SRI international (Siri = SRI), and when it was launched was meant to include ability to book restaurants etc (thereby bypassing search), but somehow those capabilities were never released and today Siri still can&#x27;t even control the iPhone!My hot take on ChatGPT plugins is a bit mixed - should be very powerful, and maybe significant revenue generator, but at same time doesn&#x27;t seem in the least bit responsible. We barely understand ChatGPT itself, and now it&#x27;s suddenly being given ability to perform arbitrary actions!

Yeah being able to generate media&#x2F;text is what excites me about these models, more than using my voice or a text input to do X instead of a webpage which has a GUI and buttons and text boxes.

This is a healthy skepticism but the difference was that using Messenger chatbots was a disjointed, clunky experience that felt slower than just a few taps in the OpenTable app. Not to mention that their natural language understanding was only marginally better than Siri at best.In this scenario, it seems dramatically faster to type or speak &quot;Find me a dinner reservation for 4 tomorrow at a Thai or Vietnamese restaurant near me.&quot; than to browse Google Maps or OpenTable. It then comes down to the quality and personalization of the results, and ChatGPT has a leg up on Google here just due to the fact that their results are not filled with ads and garbage SEO bait.

&gt;but they were fundamentally capable of the same things.This is not the case. The difference between current state of the art NLP and chatbots 3 years ago is so massive, it has to be seen as qualitative. Pre GPT-3 computers did not understand language and no commerical chatbot had any AI. Now computers can understand language.

I&#x27;m kind of skeptical of this simply because people were saying the same thing about chatbots back when there was a lot of hype around Messenger. Sure, they weren&#x27;t as advanced as what we have now, but they were fundamentally capable of the same things.Not only did the hype not pan out, but it feels as if they were completely forgotten.In a nutshell that&#x27;s why I&#x27;m still largely dismissive of anything related to GPT. It&#x27;s 2016-2018 all over again. Same tech demos. Same promises. Same hype. I honestly can&#x27;t see the big fundamental breakthroughs or major shifts. I just see improvements, but not game-changing ones.

Yes it will change the application layer. LLM allows using NUI as the universal interface to invoke under-utilized data &amp; apis. We can now develop super-app rather than many one-off apps. I have been exploring this idea since 2021, love to connect with anyone who wants to work in this space.

Why is it surprising? The amount of CPU resources server side to work on a billion iOS devices at any sort of performance level is extreme.The limitations on making Siri more useful is just adding and refining its intent system. It already integrates with Wolfram Alpha for instance.

I’m surprised Apple hasn’t improved siri with a model like this. Currently it’s just trash but with a GPT style model behind it you could actually get it to do things.

As long as the services do get paid, this is not much different than what we have nowGoogle gatekeeps everything currently, it s in the browser, the search button, the phone etc. Having chatbots instead of google is better

It’s an interesting idea, but I’m not convinced the average person has any interest in texting or talking out loud to their device to complete all their computing tasks. It’s slower for most things.Also I think there will be little interest in delegating that level of control to a single source for anything that’s important. For example, say I’ve got 5k to spend on home theatre gear, why on earth should I trust Shopify’s AI to suggest what I need and find the best price? The incentives aren’t in alignment.

I&#x27;m afraid that it has potential to subvert everything, looking at the plugins initiative is not hard to think like this: imagine the world where separate  websites and just browsing websites as we know it doesn&#x27;t exist, instead one is interacting with the model(s) directly to do what needs to be done - asking for news, buying new present for kids, discussing car models with selected price range etc.

There’s nothing stopping any LLM-backed chatbot from using plugins; the ReAct pattern discussed recently on HN is a general pattern for incorporating them.The main limits are that unless they are integral and trained-in (which is less flexible), each takes space in the prompt, and in any case the interaction also takes token space, all of which reduces the token space available to the main conversation.

My experience with Bard is it probably isn&#x27;t smart enough to figure out on its own how to use these. Google would probably have to do special finetuning&#x2F;hardcoding for the plugins that they want to work.

Bard is a tard so I doubt it. Google is done.

Anything preventing Bard&#x2F;etc from using these plugins as well?Would be nice to keep the ecosystem open.

Why do you think Apple would care? It came out in the Epic trial that 80%+ of App Store revenue comes from in app purchases in play to win games and buying loot boxes.Apple doesn’t make any money from OpenTable.

I&#x27;m not sure if the word &quot;subvert&quot; is right; the OS is still there, the App Store is still there, and nothing they&#x27;ve demonstrated will measurably impact revenue from these sources (the iOS App Store&#x27;s largest source of revenue, by far, is games. Some estimates put Games as like 25% of all of Apple&#x27;s revenue).I think there&#x27;s also a global challenge (actually, opportunity IS the right word here) that by-and-large the makers of operating systems aren&#x27;t the ones ahead in the language AI game right now. Bard&#x2F;Google may have been close six months ago, but six months is an eternity in this space. Siri&#x2F;Apple is so far behind that its not looking likely they can catch up. About a week ago a Windows 11 update was shipped which added a Bing AI button to the Windows 11 search bar; but Windows doesn&#x27;t really drive the zeitgeist.I wonder if 2023&#x2F;4 is the year for Microsoft to jump back into the smartphone OS game. There may finally be something to the idea of a more minimalist, smaller voice-first smartphone that falls back on the web for application experiences, versus app-first.

&gt; and a multi-modal response that will ultimately book my reservation?How is it going to do that? OpenTable&#x27;s value isn&#x27;t in the tech, a 15 yo could implement that over the weekend. Or maybe chatGPT can be put in the restaurant, and somehow figure out how to seat you. And then you&#x27;d have a human talking to chatGPT and chatGPT talking to another chatGPT to complete the task. That&#x27;ll be interesting, but otherwise this is overly complicated for all parties involved.

Everyone&#x27;s been talking about how ChatGPT will disrupt search, but looking at the launch partners, I think this has the potential to completely subvert the OS &#x2F; App Store layer. On some level, how much do I need an OpenTable app if I can use voice&#x2F;text input and a multi-modal response that will ultimately book my reservation?Not saying mobile&#x27;s going away, but this could be the thing that does to mobile what mobile did to desktop.

I&#x27;m so hyped for the ChatGPT-4 API. Wish they&#x27;d give me access so I can make a lot of my workflows much easier. Especially in terms of translations.

I wonder how you pay for it?Are the plugins going to cost more?Do they share the $20 with the plug provider?do you get charged a pay per use?

It reminds me of what went down with Netflix. At first, it looked like you only needed one subscription to watch everything, but now that other players have entered the market, with their own bussiness contacts we&#x27;re seeing ecosystems fracture.For example, Microsoft is collecting data from services A, B, and C, while Google is gathering data from X, Y, and Z. And when it comes to language models, you might use GPT for some tasks and Llama or Bard for others. It seems like the fight ahead won&#x27;t be about technology, but rather about who has access to the most useful dataset.Personally, I also think we&#x27;ll see competitors trying to take legal action against each other soon.

1) Not every use cases will require the full power and (probably) considerable cost of chat GPT-4.2) some companies can absolutely not use OpenAI tools simply because they are American and online. A competitor might emerge to capture that market and be allowed to grow to be &quot;good enough&quot;3) some &quot;countries&quot; (think China or EU(who am I Kidding)) will limit their growth until local alternatives are available. Ground breaking technology have a tendency to spread globally and the current state of the art is not that expensive (we are talking single digit billions once)

The groups are focused on different things.OpenAI went all in on generative models, i.e. stable diffusion and large language models. DeepMind focused on reinforcement learning, tree search, plus alphafold approaches to biology. FAIR has translation, pytorch, and some LLM stuff in biology.What OpenAI is missing though is any AI research in biology, but I bet they are working on it.I&#x27;m not sure if this makes sense but OpenAI seems to be operating at a higher level of abstraction (AGI) where they are integrating modalities (text and image modality for now, probably speech next) vs the other places have taken a more focused applied approach.

Google have really been caught with their pants down here.Remember that OpenAI was created specifically to stave off the threat of AI monopolization by Google (or anyone else - but at the time Google).DeepMind have done some interesting stuff with Go, Protein folding etc, but nothing really commercial, nor addressing their reason d&#x27;etre of AGI.Google&#x27;s just-released ChatGPT competitor, Bard, seems surprisingly weak, and meantime OpenAI are just widening their lead. Seems like a case of the small nimble startup running circles around the big corporate behemoth.

How about training data from interactions just by sheer usage numbers? Google does not have that.There is a reason why the quality of ChatGPT responses are better. RLHF.I am not sure though how Google can be 3x better than OpenAI to make user switch now. They are so slow, they should be the one building the plugins.

I don&#x27;t see much of a moat currently, or even developer lockin. The current APIs, and this new plugin architecture are dead simple.

I&#x27;ve got to wonder, how does a second player in the LLM space even get on the board?Like, this feels a lot like when the iPhone jumped out to grab the lion share of mobile. But the switching costs was much smaller (end users could just go out and buy an Android phone), and network effects much weaker (synergy with iTunes and the famous blue bubbles... and that&#x27;s about it). Here it feels like a lot of the value is embedded in the business relationships OpenAI&#x27;s building up, which seem _much_ more difficult to dislodge, even if others catch up from a capabilities perspective.

Get this thing running in a tight loop with an internal monologue in a car and you&#x27;ll mostly solve self driving.

The difference is unlike self-driving and crypto, LLMs are providing value to people _today_.In my personal life, GPT4 is a patient interlocutor to ask about nerdy topics that are annoying to google (eg, yesterday I asked it &quot;What&#x27;s the homologue of the caudofemoralis in mammals?&quot;, and a long convo about the subtleties of when it is and isn&#x27;t ok to use &quot;gè&quot; as the generic classifier in Mandarin.)Professionally, it&#x27;s great for things like &quot;How do I recursively do a search and replace `import &quot;foo&quot; from &quot;bar&quot;` to `import &quot;baz&quot; from &quot;buzz&quot;`, or &quot;Pull out the names of functions defined in this chunk of scala code&quot;. This is without tighter integrations like Copilot or the ones linked to above.

NFTs never provided a single use case. It was always some bullshit to pretend it&#x27;s valuable to rugpull people.ChatGPT is useful today for real use cases. It&#x27;s tangible!

It&#x27;s really over the top hype the likes of which we haven&#x27;t seen since self driving, blockchain&#x2F;bitcoin, etc.  I suspect in a year there will be some interesting uses of LLMs but all of the &#x27;this changes EVERYTHING&#x27; pie in the sky thinking will be back down to earth.

Seems like someone already wrote an HN plugin. More than one enthusiastic comment per minute on this thread and it was just posted half an hour ago. Plus HN is filled with enthusiasm about ChatGPT today. Seems sus.

Is there a plugin to automate signing up for waitlists? That&#x27;s what I&#x27;ve needed this week.

Stable Diffusion A1111 and other webUIs are moving so fast with a bunch of OSS contributions, seems pretty rational for OpenAI to decide to not compete and just copy the interfaces of the popular tools once the users validate their usefulness rather than trying to design them a priori.

What&#x27;s wrong with being wrong? In this case, I&#x27;m delighted to be wrong (though I believe I had evaluated OpenAI mostly right given my knowledge at the time).

I rather disagree!Writing and discussion are great ways to explore topics and crystallize opinions and knowledge. HN is a pretty friendly place to talk over these earth moving developments in our field and if I participate here, I’ll be more ready to participate when I get asked if we need to spin up an LLM project at work.

As long as your opinions&#x2F;predictions are backed by well-reasoned arguments, you shouldn&#x27;t be afraid to share them just because they might turn out to be wrong. You can learn a lot by having your arguments rebutted, and in the end no one really cares one way or the other.Just don&#x27;t end up like that guy who predicted that Dropbox would never make it off the ground... that was not a well-reasoned position.

Though there might be nothing beneficial about being right or getting upvotes, and it is easy to be wrong, an important thing on a forum like this is the spread of new ideas. While someone might be hesitant to share something because it&#x27;s half-baked, they might have the start to an idea that could have some discussion value.

For me, this is why I hesitate to comment and write significant, lengthy comments on here, or any website. It&#x27;s easy to be wrong (like you), and while being wrong isn&#x27;t bad, there isn&#x27;t necessarily any upside to being right either, aside from the dopamine rush of getting upvotes, which in life, doesn&#x27;t amount to much.

Hopefully they do not feel free the pressure to &quot;move fast and break things&quot; because this in turn has the ability to &quot;move fast and break everything&quot;

Agreed. This makes me realize that OpenAIs leadership is able to look long term and decide where to properly invest, as most of the decisions to take the projects in these directions were made &gt;1 year ago.One can only wonder what they’re working on at this very moment.

You’re right thoughDisclosure: I work for Google

Is it better than Stable Diffusion 1.5?Disclosure: No job, do whatever I want.

It&#x27;s so good, I&#x27;m pretty sure it is Dall-E 3. Probably Microsoft negotiated a few weeks of exclusivity, like with GPT-4.By the way, Microsoft made it completely free to use. Surprised it isn&#x27;t discussed much.

Then again, the new DALL-E model just released in Bing Chat is really good.Disclosure: I work at Microsoft.

Ironic given OpenAI&#x27;s initial messaging explicitly was:&quot;OpenAl is a non-profit artificial
intelligernce research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.&quot;Ultimately govt is an idiosyncratic, capricious (and sometimes corrupt?) God of what Nassem Taleb would call Hidden Asymmetries&quot;; as in the case of which elonymous company ingests massive tax credits; or which banks get to survive etc

I like how you assume that a non-profit organization is hell-bent on monopolizing a market. What you wrote sounds made up, do you have any sources?

No that wasn&#x27;t what they had in mind at all, it was pretty clear from the start that they intended to monetize DALL-E. It&#x27;s just that it turned out that you require far smaller models to be able to do generative art, so competitors like stability AI were able to release viable alternatives before OpenAI could establish a monopoly.Why do you think that Sam Altman keeps calling for government intervention with regards to AI? He doesn&#x27;t want to see a repeat of what happened with generative art, and there&#x27;s nothing like a few bureaucratic road blocks to slow down your competitors.

A couple (wow, only 5!) months ago, I wrote up this long screed[1] about how OpenAI had completely missed the generative AI art wave because they hadn&#x27;t iterated on DALL-E 2 after launch. It also got a lot of upvotes which I was pretty happy about at the time :)Never have I been more wrong. It&#x27;s clear to me now that they simply didn&#x27;t even care about the astounding leap forward that was generative AI art and were instead focused on even more high-impact products. (Can you imagine going back 6 months and telling your past self &quot;Yeah, generative AI is alright, but it&#x27;s roughly the 4th most impressive project that OpenAI will put out this year&quot;?!) ChatGPT, GPT4, and now this: the mind boggles.Watching some of the gifs of GPT using the internet, summarizing web pages, comparing them, etc is truly mind-blowing. I mean yeah I always thought this was the end goal but I would have put it a couple years out, not now. Holy moly.[1]: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=33010982" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=33010982</a>

OpenAI is like a virus... the speed at which it degrades its competitors is staggering.

Super smart move for OpenAI to monetize the existing infrastructure, which will make it easy for corporations to integrate GPT into their internal data and workflow. It also solves two fundamental bottlenecks in current versions of GPT: factuality and (limited) working memory. Google, with its lackluster Bard, will face new threat, now that everyone can build a customized New Bing clone in a matter of days.

Wow. GPT-4 have already become kind of my personal assistant in the last couple of weeks, and now it will be able to actually perform tasks instead of just giving me text descriptions.

&gt; &quot;We expect that open standards will emerge to unify the ways in which applications expose an AI-facing interface. We are working on an early attempt at what such a standard might look like, and we’re looking for feedback from developers interested in building with us.&quot;I&#x27;m curious to see just how they&#x27;re going to play this &quot;open standard.&quot;

Agree for me it probably looks similar to situation with iphone history - first one was impressive but only when next year after that  apple released app store they turned snow ball rolling into unstoppable avalanche.

Hopefully it doesn’t actually become THE “killer” app

This changes everything and seems like a perfect logical step from where we were. LLMs have this fantastic capacity to understand human language, but their abilities were severely limited without access to the external world. Before, I felt ChatGPT was just a cool toy. Now that ChatGPT has plugins, the sky’s the limit. I think this could the “killer app” for LLMs.

re money, people are falling over themselves to pay money for this thing and they&#x27;re being put on a waitlist.this thing seems to be like cellphones, everyone will need a subscription or you&#x27;re an outcast or something.

The problem is, this will have downstream effects. Google funnels people onto third party websites and these third party websites are able to sustain themselves thanks to the ad revenue they make from traffic. We need other players to make money other than the middleman.

Google is currently in an existential crisis on this front... Microsoft is already way ahead of the game when it comes to integrating LLMs into productivity tools &amp; search. This recent product announcement about Microsoft 365 integration is almost magical:<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=Bf-dbS9CcRU">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=Bf-dbS9CcRU</a>Best of all: Advertising needn&#x27;t be the business model! And Microsoft is a major investor &#x2F; partner for OpenAI.

Thought 1:
If google can get their shit together and actually integrate their LLM with all their services and all the data they have they would have a strong edge over the competition. An LLM that can answer questions based on your calendar, your email, your google docs, youtube&#x2F;search history, etc. is simultaneously terrifying and interesting.Of course there&#x27;s also microsoft who does have some popular services, but they&#x27;re pretty limited.Thought 2:
How do these companies make money if everyone just uses the chatbot to access them? Is LLM powered advertising on the way?

This goes in line with the “Open” in OpenAI. However, this is a &quot;controlled&quot; sort of openness, and the problem of trust with their receding &quot;real&quot; openness does not encourage me to engage with this ecosystem.

go home and cry bill, you&#x27;re drunk again

I have a plan: let&#x27;s blame the FED and save the VCs

What&#x27;s that noise?That&#x27;s the sound of a thousand small startups going bust.Well played OpenAI.

ChatGPT Plugins