Indeed! Shame there&#x27;s a lack of access to ultra for now, but good to have more things to access.Also:&gt; Starting today, Bard will use a fine-tuned version of Gemini Pro for more advanced reasoning, planning, understanding and more. This is the biggest upgrade to Bard since it launched.edit-Edit 2 - forget the following, it&#x27;s not available here but that&#x27;s hidden on a support page, so I&#x27;m not able to test it at all.Well that&#x27;s fun. I asked bard about something that was in my emails, I wondered what it would say (since it no longer has access). It found something kind of relevant online about someone entirely different and said&gt; In fact, I&#x27;m going to contact her right now

Yep. That&#x27;s their &quot;moat&quot;, to go with The Discourse. For better or for worse, a bunch of us know how to use their models, where the models do well, where the models are a little rickety, etc. Google needs to build up that same community.

OpenAI did well to let anyone try it with a login on a website.

I still think it&#x27;s worth it. GPT-3.5 is extremely powerful, and it&#x27;s what we use in production. GPT-4 is way overkill for our prompt and use case.If it&#x27;s similar, or even marginally better in any way, we&#x27;d consider switching over. Not because OpenAI is bad or anything (they&#x27;re great, actually!) but because it&#x27;s so easy to do that.

Gemini Pro is only GPT3.5 tier according to the benchmarks, so unless they make it extremely cheap I don&#x27;t see much value in even playing around with it

&gt; Starting on December 13, developers and enterprise customers can access Gemini Pro via the Gemini API in Google AI Studio or Google Cloud Vertex AI.Excited to give this a spin. There will be rough edges, yes, but it&#x27;s always exciting to have new toys that do better (or worse) in various ways.

It&#x27;s on par with GPT3.5, assuming they didn&#x27;t overtrain it to pass the tests.

Bard will now be using Gemini Pro. I&#x27;m excited to check it out

And the technical report: <a href="https:&#x2F;&#x2F;storage.googleapis.com&#x2F;deepmind-media&#x2F;gemini&#x2F;gemini_1_report.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;storage.googleapis.com&#x2F;deepmind-media&#x2F;gemini&#x2F;gemini_...</a>

Much more interesting link: <a href="https:&#x2F;&#x2F;deepmind.google&#x2F;technologies&#x2F;gemini&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;deepmind.google&#x2F;technologies&#x2F;gemini&#x2F;</a>

Google Search : Did you mean 800 pound gorilla?

I’m curious which instances of overpromising you’re referring to.

Everything they published thus far in the generative AI space has been abysmal in quality compared to the competition. I&#x27;d be hella surprised if this reaches GPT-4 levels of quality...

&gt;Finally, some competition for GPT4 API!!! This is such good news.Save your enthusiasm for after it launches; Google&#x27;s got a habit of over-promising when it comes to AI.

&gt;So they are roughly a year behind OpenAIPossibly by that time GPT5 will already be out.

&gt; and that is assuming their model is not overtrained to just pass the tests slightly better than GPT4You are assuming GPT4 didn&#x27;t do the exact same!Seriously, it&#x27;s been like this for a while, with LLMs any benchmark other than human feedback is useless. I guess we&#x27;ll see how Gemini performs when it&#x27;s released next year and we get independent groups comparing them.

It won&#x27;t be available to regular devs until Q2 next year probably (January for selected partners). So they are roughly a year behind OpenAI - and that is assuming their model is not overtrained to just pass the tests slightly better than GPT4

&quot;Google, for better or worse, has more of a brand reputation to maintain&quot;You think this is why Google is so far behind?

Heh, I read that and had the opposite conclusion.When I was reading the benchmarks and seeing how Gemini Ultra was outperforming GPT-4 I thought, &quot;Finally, some competition for GPT4&quot;!But when I got to that part, that&#x27;s when I realized that it could potentially be caught in release hell and not actually see the light of day or significant use. Google, for better or worse, has more of a brand reputation to maintain and is more risk averse, so even if Gemini Ultra can, in theory, outperform GPT4, users might not get a chance to access it for a while.

I’m afraid it won’t be nearly as good as GPT4, because of how lax Open AI can be with intellectual property. Google will not be able to train their model on Libgen or Opensubtitles, because they can’t afford the risk.

&gt; presumably GPT4 will have grown as much as wellMost of the comments I see on Hacker News claim ChatGPT is getting worse at different things (though I don&#x27;t believe those claims).

Even if they&#x27;re perpetually a year behind, that&#x27;s a strong competitive spur to keep OpenAI on the move.

Well not exactly. Not coming out until later when presumably GPT4 will have grown as much as well. So far each time, Google has failed to catch up to OpenAI. Hopefully they do however eventually.

That&#x27;s actually not a bad achievement considering Google is now the new IBM.

won&#x27;t be available for regular devs until probably Q2 next year, OpenAI will have probably released GPT5 or whatever new model by then. And GPT4 was done training in 2022, the fact Google is thumping their chest about being 2 years behind a much smaller company is kind of pathetic

&gt; 
For Gemini Ultra, we’re currently completing extensive trust and safety checks, including red-teaming by trusted external parties, and further refining the model using fine-tuning and reinforcement learning from human feedback (RLHF) before making it broadly available.&gt; As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for early experimentation and feedback before rolling it out to developers and enterprise customers early next year.Finally, some competition for GPT4 API!!! This is such good news.

<a href="https:&#x2F;&#x2F;youtu.be&#x2F;LvGmVmHv69s" rel="nofollow noreferrer">https:&#x2F;&#x2F;youtu.be&#x2F;LvGmVmHv69s</a>

They certainly claim it is SOTA for multimodal tasks: “Gemini surpasses SOTA performance on all multimodal tasks.”

Usually SOTA status is established when the benchmark paper is released (probably after some review). But GPT4 is the current generally-available-SOTA

They also compare to RLHFed GPT-4, which reduces capabilities, while their model seems to be pre-RLHF. So I&#x27;d expect those numbers to be a bit inflated compared to public release.

It will be SOTA maybe when Gemini Ultra is available. GPT-4 is still SOTA.

They&#x27;ve reported surpassing GPT4 on several benchmarks. Does anyone know of these are hand picked examples or is this the new SOTA?

GPT-4 had finished training 8 months prior to release.

Yeah and GPT-4V is even younger (at least it was available later)

GPT-4 launched a little less than 9 months ago.

The new alternatives offered better products. Not clear that Gemini qualifies yet besides multimodal.

Less than a year difference is &quot;quite far behind&quot;?Lotus 1-2-3 came out 4 years before Microsoft Excel. WordPerfect came out 4 years before Microsoft Word.Hotmail launched 8 years before Gmail. Yahoo! Mail was 7 years before Gmail.Heck, AltaVista launched 3 years before Google Search.I don&#x27;t think less than a year difference is meaningful at all in the big picture.

Gemini looks like a better GPT-4 but without the frequent outages.

This is interesting in that it implies that catching up is possible if you have enough data, engineers and compute. This also potentially implies that adjacent players such as Nvidia could gain an edge long term because they are a leader in one of the three.

So it&#x27;s basically just GPT-4, according to the benchmarks, with a slight edge for multimodal tasks (ie audio, video).
Google does seem to be quite far behind, GPT-4 launched almost a year ago.

Well they sure copied Apple&#x27;s &quot;Pro&quot; and &quot;Ultra&quot; branding. I&#x27;m fully expecting a &quot;Gemini Max&quot; version in the near future!

This marketing page feels very apple-like (and I mean that in a good way).If the benchmarks are any indication, Gemini seems legit, excited to see what it can do.

Bard is apparently based on gemini pro from today, pro is coming via api on the 13th and ultra is still in more &quot;select developers&quot; starting next year.

It&#x27;s a shame that Gemini Ultra is not out yet, it seems like a solid improvement on GPT-4. I wonder how it&#x27;ll compare against GPT-5?

There are some 7B weight models that look competitive with GPT4 on benchmarks, because they were trained on the benchmark data. Presumably Google would know better than to train on the benchmark data, but you never know. The benchmarks also fail to capture things such as Bard refusing to tell you how to kill a process on Linux because it&#x27;s unethical.

The articles seems to report some data points which at least make it seem comparable to GPT4. To me, I feel as though this makes it more objective vs fluff.

Not anymore, it&#x27;d be &quot;Apple AI+&quot; or &quot;Siri+&quot;.

Maybe if it was still 2010 you would be correct.

At least Apple would call it iParrot or iSomething. :D

I wonder what % of google shareholders understand this difference you talk about between “available now! try it” and “available Q2 next year”.

At least they can claim SOTA with this, even if their product remains unavailable. Let&#x27;s Google still appear competitive even if GPT-5 beats it and is publicly available before Gemini

I&#x27;m not a marketer but it&#x27;s hard to see what the point of these glossy press releases without a call to action is.If I could have tried it today, I would have dropped everything and tried it. Now I will forget about it for a while and try it whenever I hear through osmosis that it&#x27;s available. To the extent that I am excited and&#x2F;or interested, the snooze button is pressed.Maybe that&#x27;s the desired outcome?

They have even adopted some of the Apple nomenclature &quot;Ultra&quot;, &quot;Pro&quot; and &quot;Nano&quot;

That really does look like something you&#x27;d see if you went to Apple.com

Look at their landing page: <a href="https:&#x2F;&#x2F;deepmind.google&#x2F;technologies&#x2F;gemini&#x2F;#introduction" rel="nofollow noreferrer">https:&#x2F;&#x2F;deepmind.google&#x2F;technologies&#x2F;gemini&#x2F;#introduction</a>It&#x27;s like they hired Apple to do their marketing.

Feels more like an Apple post &quot;the best fastest blabla-est&quot;. How about making it available to try without the fluff?

It&#x27;s funny the page says BLUE score instead of BLEU score.I bet it started off as BLEU and then during the editing process it got &#x27;corrected&#x27; to BLUE.

Wasn&#x27;t there a news sometimes before that Sundar and Demis didn&#x27;t get along. Only after ChatGPT, Sundar got orders from above to set house in order and focus everything on this and not other fundamental research projects which Demis likes to work on.

Gemini can become a major force with 7% increase in code-writing capability when GPT-4 is getting lazy about writing code these days.Better OCR with 4% difference, better international ASR, 10% decrease.Seeing Demis Hassabis name in the announcement makes you think they really trust this one.

isn&#x27;t that the definition of diminishing returns? just asking - that&#x27;s how I always interpreted that phrase...

Depends on if you mean &quot;better&quot; as better score (5% better) or &quot;better&quot; as in &quot;fewer errors&quot; (100% better).

Wouldn&#x27;t 95% vs 90% mean 2x better, not 5% better?

&gt; We’re already starting to experiment with Gemini in Search, where it&#x27;s making our Search Generative Experience (SGE) faster for users, with a 40% reduction in latency in English in the U.S., alongside improvements in quality.This feels like Google achieved a more efficient inference. Probably a leaner model wrt GPT.

not sure, but you could also look at the inverse. e.g. a 90% to 95% improvement could also be interpreted as 10% failure to 5% failure, i.e. half the amount of failures, a very big improvement. It depends on a lot of things, but it&#x27;s possible that this could feel like a very big improvement.

Training large language models is characterised by diminishing returns; the first billion training inputs reduce the loss more than the second billion, the second billion reduce the loss more than the third, etc. Similar for increases in size; the improvement is less than linear.

It may mean that the evaluations useful range of distinguishing inprovements is limited. If its a 0-100 score on defined sets of tasks that were set because they were hard enough to distinguish quality in models a while back, the rapid rate of improvement may mean that they are no longer useful in distinguishing quality of current models even aside from the problem that it is increasingly hard to stop the actual test tasks from being reflected in training data in some form.

Probably just reflects that they are playing catch-up with OpenAI, and it would not look good if they announced their latest, greatest (to be available soon) was worse that what OpenAI have been shipping for a while, so I assume that being able to claim superiority (by even the smallest amount) over GPT-4 was the gating factor for the this announcement.I doubt LLMs are close to plateauing in terms of performance unless there&#x27;s already an awful lot more to GPT-4&#x27;s training than is understood. It seems like even simple stuff like planning ahead (e.g. to fix &quot;hallucinations&quot;, aka bullshitting) is still to come.

They want to release immediately to please shareholders but only if they&#x27;re beating SOTA in benchmarks. Therefore we will usually get something which beats SOTA by a little bit, because the alternative (aside from a huge breakthrough) would be to delay release longer which serves no business purpose.

The improvement over ChatGPT are counted in (very) few percents. Does it mean they have entered a diminishing returns phase or is it that each percent is much harder to get compared to the previous ones ?

They better. I haven’t used google search in a while.

This is Android moment for Google. They will go full throttle on it till they become dominant in every respect.

Sure, but it doesn’t mean that it stops being surprising. It’s like a “time is relative” kind of thing for organizational logic.Imagine an organization on the scale of Google, with everything in it’s favor, being outmaneuvered by a much smaller one in such a transcendental endeavor. It’s like to a small country in Central America, coming up with some weapon to rival the US’s army.

Maybe small teams can be faster than huge teams?

It’s impressive, but we know that there’s a lot more than just that.

How many other companies can you say that have possibly passed GPT-4?

It&#x27;s truly astounding to me that Google, a juggernaut with decades under its belt on all things AI, is only now catching up to OpenAI which is on all camps a fraction of its size.

yeah a lot of local models fall short on that benchmark as well. I wonder what was different about GPT3.5&#x2F;4&#x27;s training&#x2F;date that would lead to its great hellaswag perf

Tech report seems to hint at the fact that GPT-4 may have had some training&#x2F;testing data contamination and so GPT-4 performance may be overstated.

I wonder how that weird HellaSwag lag is possible. Is there something really special about that benchmark?

The performance results here are interesting. G-Ultra seems to meet or exceed GPT4V on all text benchmark tasks with the exception of Hellaswag where there&#x27;s a significant lag, 87.8% vs 95.3%, respectively.

Ya, I was expected a larger improvement in math related tasks with Gemini.

Google has the biggest proprietary moat of information of any company in the world I&#x27;m sure.

OpenAI had an almost five-year head-start with relevant data acquisition and sorting, which is the most important part of these models.

And GPT-4 finished training summer 2022 and OpenAI was sitting on it for like another 9 months

GPT-4 is also rumored to have consumed 5x less compute to train

This is very cool and I am excited to try it out!But, according to the metrics, it barely edges out GPT-4 -- this mostly makes me _more_ impressed with GPT-4 which:- came out 9 months ago AND- had no direct competition to beat (you know Google wasn&#x27;t going to release Gemini until it beat GPT-4)Looking forward to trying this out and then seeing OpenAI&#x27;s answer

Do you have evidence of that? Photoshop has blocked you from editing pictures of money for ages and that wasn&#x27;t in the cloud. Moreover, how does a Google data center know whether you&#x27;re allowed to swap a particular face versus your device? It&#x27;s quite a reach to assume Google would go out of their way to prevent you from doing things on your device in their app when other AI-powered apps on your device already exist and don&#x27;t have such policy restrictions.

I think a lot of the motivation for running it in the cloud is so they can have a single point of control for enforcing editing policies (e.g. swapping faces).

Yeah, even more exciting than the small PaLM2 from Google IO

The fact that it&#x27;s multimodal is very interesting, they might not make it open source, but if they intend run it on people&#x27;s devices, even if they intend to implement DRM, someone will figure out how to extract the weights and get it running outside.

Unless they make it open source, what benefits would it have compared to the tiny OpenAi models?

It runs an on-device LLM to generate a HTTP POST every time. It took four interns half a week to reduce the hallucinations, but a PM got a promotion after that.

Using the power of a Bugatti Veyron, opens and closes the garage door.

&gt; &quot;Using the power of Google Tensor G3, Video Boost on Pixel 8 Pro uploads your videos to the cloud where our computational photography models adjust color, lighting, stabilization and graininess.&quot;*I wonder why the power of Tensor G3 is needed to upload your video to the cloud...*<a href="https:&#x2F;&#x2F;blog.google&#x2F;products&#x2F;pixel&#x2F;pixel-feature-drop-december-2023&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;blog.google&#x2F;products&#x2F;pixel&#x2F;pixel-feature-drop-decemb...</a>

Gemini Nano sounds like the most exciting part IMO.IIRC Several people in the recent Pixel 8 thread were saying that offloading to web APIs for functions like Magic Eraser was only temporary and could be replaced by on-device models at some point. Looks like this is the beginning of that.

Google&#x27;s embargo seemed to relate to their battle with the Canadian government over news. Given that they settled on that I&#x27;d expect its availability very soon.Anthropic is a bit weird and it almost seems more like lazy gating. It&#x27;s available in the US and UK, but no EU, no Canada, no Australia.

Right but Bard is literally available in 230 countries and territories...but not Canada.<a href="https:&#x2F;&#x2F;support.google.com&#x2F;bard&#x2F;answer&#x2F;13575153?hl=en#:~:text=Bard%20is%20currently%20available%20in,regulations%20and%20our%20AI%20principles" rel="nofollow noreferrer">https:&#x2F;&#x2F;support.google.com&#x2F;bard&#x2F;answer&#x2F;13575153?hl=en#:~:tex...</a>.We are being singled out because of the Government&#x27;s Online News Act for tech companies to pay for news links

Perhaps they&#x27;re being cautious after <a href="https:&#x2F;&#x2F;www.reuters.com&#x2F;technology&#x2F;canada-launch-probe-into-openai-over-privacy-concerns-2023-05-25&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.reuters.com&#x2F;technology&#x2F;canada-launch-probe-into-...</a> ?

Anthropic&#x27;s Claude is still not available in Canada either. Anyone have insight into why its difficult to bring these AI models to Canada when on the surface its political and legal landscape isn&#x27;t all that different from the US?

I&#x27;m in Canada too I just fired up Tunnel Bear VPN and was able to see bard OK.But yeah weird we are usually lumped in with the US market.

Quite surprising to me. Bard has been available in Pakistan for a couple of months I believe.

Of course you can. Even I can use it here in Russia.

There are also countries like the uk that get bard but without gemini but with the old palm model instead.<a href="https:&#x2F;&#x2F;support.google.com&#x2F;bard&#x2F;answer&#x2F;14294096?visit_id=638374811255044310-1362936850&amp;p=gemini_pro_countries&amp;rd=1" rel="nofollow noreferrer">https:&#x2F;&#x2F;support.google.com&#x2F;bard&#x2F;answer&#x2F;14294096?visit_id=638...</a>

It&#x27;s likely due to regulatory compliance.&gt;Canadian lawmakers recently introduced legislation aimed at regulating AI. The Artificial Intelligence and Data Act (AIDA) mandates assessments, risk management, monitoring, data anonymization, transparency, and record-keeping practices around AI systems. AIDA would also introduce penalties of up to 3% of a company’s global revenue or $10 million.These idiots don&#x27;t seem to realize that a VPN bypasses all of their silly compliance BS.

Bard still not available in Canada so i can&#x27;t use it ¯\_(ツ)_&#x2F;¯. Wonder why Google is the only one that can&#x27;t release their model here.

Yep, the announcement is quite cheeky.Ultra is out sometime next year, with GPT-4 level capability.Pro is out now (?) with ??? level capability.

Yep, at this point I&#x27;d rather they hold their announcements until everybody can access it, not just the beautiful people.  I&#x27;m excited and want to try it right now, and would actually use it for a PoC I have in mind, but in a few months the excitement will be gone.

The article says &quot;next year&quot; - so that could be as soon as January, right?

I hate this &quot;tierification&quot; of products into categories: normal, pro, max, ultraApple does this and it&#x27;s obvious that they do it to use the &quot;decoy effect&quot; when customers want to shop. Why purchase a measly regular iPhone when you can spend a little more and get the Pro version?But when it comes to AI, this tierification only leads to disappointment—everyone expects the best models from the FAANGO (including OpenAI), no one expects Google or OpenAI to offer shitty models that underperform their flagships when you can literally run Llama 2 and Mistral models that you can actually own.

There was a waiting period for ChatGPT4 as well, particularly direct API access, and the WebUI had (has?) a paywall

That&#x27;s for Ultra right? Which is an amazing accomplishment, but it sounds like I won&#x27;t be able to access it for months. If I&#x27;m lucky.

That&#x27;s because they&#x27;re not &quot;scientific papers&quot;, they&#x27;re technical papers.

It&#x27;s aimed at the AI pontification industrial complex.

If it feals like a duck it is a duck. Duh(ck)

I miss when ML scientific papers had actual science in them. Now they all feel like ads.

It really feels like the reason this is being released now and not months ago is that that&#x27;s how long it took them to figure out the convoluted combination of different evaluation procedures to beat GPT-4 on the various benchmarks.

Why is that misleading? It shows Gemini with CoT is the best known combination of prompt and LLM on MMLU.They simply compare the prompting strategies that work best with each model. Otherwise it would be just a comparison of their response to specific prompt engineering.

The places where they use the same methodology seem within the error bars of the cherry picked benchmarks they selected. Maybe for some tasks it&#x27;s roughly comparable to GPT4 (still a major accomplishment for Google to come close to closing the gap for the current generation of models), but this looks like someone had the goal of showing Gemini beating GPT4 in most areas and worked back from there to figure out how to get there.

The table is *highly* misleading.
It uses different methodologies all over the place.For MMLU, it highlights the CoT @ 32 result, where Ultra beats GPT4, but it loses to GPT4 with 5-shot, for example.For GSM8K it uses Maj1@32 for Ultra and 5-shot CoT for GPT4, etc.Then also, for some reason, it uses different metrics for Ultra and Pro, making them hard to compare.What a mess of a &quot;paper&quot;.

Some discussion on twitter about misleading evaluation: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;a_a_cabrera&#x2F;status&#x2F;1732454328307511807" rel="nofollow noreferrer">https:&#x2F;&#x2F;twitter.com&#x2F;a_a_cabrera&#x2F;status&#x2F;1732454328307511807</a>(nitter: <a href="https:&#x2F;&#x2F;nitter.net&#x2F;a_a_cabrera&#x2F;status&#x2F;1732454328307511807#m" rel="nofollow noreferrer">https:&#x2F;&#x2F;nitter.net&#x2F;a_a_cabrera&#x2F;status&#x2F;1732454328307511807#m</a>)

I wish Google’s UI would have the same chat interface as ChatGPT or even command line ones I’ve encountered

So, better than GPT4 according to the benchmarks? Looks very interesting.Technical paper: <a href="https:&#x2F;&#x2F;goo.gle&#x2F;GeminiPaper" rel="nofollow noreferrer">https:&#x2F;&#x2F;goo.gle&#x2F;GeminiPaper</a>Some details:- 32k context length- efficient attention mechanisms (for e.g. multi-query attention (Shazeer, 2019))- audio input via Universal Speech Model (USM) (Zhang et al., 2023) features- no audio output? (Figure 2)- visual encoding of Gemini models is inspired by our own foundational work on Flamingo (Alayrac et al., 2022), CoCa (Yu et al., 2022a), and PaLI (Chen et al., 2022)- output images using discrete image tokens (Ramesh et al., 2021; Yu et al., 2022b)- supervised fine tuning (SFT) and reinforcement learning through human feedback (RLHF)I think these are already more details than what we got from OpenAI about GPT4, but on the other side, still only very little details.

This also jumped out at me. It also seems that they are selectively choosing different promoting strategies too, one lists &quot;CoT@32&quot;.Makes it seem like they really needed to get creative to have it beat GPT4. Not a good sign imho

I assume these landing pages are made for wall st analysts rather than people who understand LLM eval methods.

Page 7 of their technical report [0] has a better apples to apples comparison. Why they choose to show apples to oranges on their landing page is odd to me.[0] <a href="https:&#x2F;&#x2F;storage.googleapis.com&#x2F;deepmind-media&#x2F;gemini&#x2F;gemini_1_report.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;storage.googleapis.com&#x2F;deepmind-media&#x2F;gemini&#x2F;gemini_...</a>

What is up with that eval @32? Am I reading it correctly that they are generating 32 responses and taking majority? Who will use the API like that? That feels like such a fake way to improve metrics

just as a quick sanity check, it manages to solve day 1 part 1 of advent of code, same as chatgpt4.Notably it also solves _part 2_ which chatgpt4 struggled with.

&quot;What the quack&quot; one really got me.

The hands-on demo is pretty cool. Need this on phone asap.<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI</a>

This demo video makes it seem like it would have a decent shot<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI</a>

Killing ad free internet is good for google shareholders. That’s the “everyone” they’re talking about in case it wasn’t clear.

Wasn&#x27;t Gemini part of Greek Mythology way, way before?  Aren&#x27;t you losing maybe thousands of years here?

It probably is a coincidence. But as-per my other comment, an unfortunate one.Take all the hundreds of thousands of words in popular languages. And all the human names. And all possible new made up words and made up names. And land on one that&#x27;s a project with a FAQ[1] saying &quot;Gemini might be of interest to you if you: Value your privacy and are opposed to the web&#x27;s ubiquitous tracking of users&quot; - wait, that&#x27;s Google&#x27;s main source of income isn&#x27;t it?[1] <a href="https:&#x2F;&#x2F;geminiprotocol.net&#x2F;docs&#x2F;faq.gmi" rel="nofollow noreferrer">https:&#x2F;&#x2F;geminiprotocol.net&#x2F;docs&#x2F;faq.gmi</a>

It&#x27;s probably just an unfortunate coincidence. After all, Gemini is a zodiac sign first and foremost, you&#x27;d have to specify what exactly you want anyway.

It&#x27;s not just the confusion part, it&#x27;s the behaviour part; Google is a web browser near-monopolist, AI is a big hyped thing with a lot of media noise, and Google has plonked their AI right on top of a small competitor, and done it while crowing about how moral they are.It&#x27;s more like Uber releasing an AI called Taxi so all web searches for taxis show you Uber results and you saying &quot;taxi is a word used by lots of companies&quot;.

Maybe they shouldn&#x27;t have chosen such a common word if they didn&#x27;t want to be confused with something else. <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Gemini" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Gemini</a>

&gt; &quot;Gemini as a web protocol isn&#x27;t even on the top 5 list of things that come up when you think about Gemini prior to this announcement.&quot;And now it never will be :)

Gemini as a web protocol isn&#x27;t even on the top 5 list of things that come up when you think about Gemini prior to this announcement. It would be surprising if anyone involved in naming the Google product even knew about it.

There&#x27;s some dissonance in the the way this will swamp out searches for the web-alternative Gemini protocol by the biggest tech company in the world proudly boasting how responsible and careful they are being to improving things &quot;for everyone, everywhere in the world&quot;.

I&#x27;ve been asking Bard and it&#x27;s telling me it&#x27;s latest major update was September  and it&#x27;s backend is LaMDA... not sure if that means anything though

The Pro seem to be available in Bard already.

I&#x27;ve missed this on my initial skim:The one launching next week is Gemini Pro.The one in the benchmarks is Gemini Ultra which is &quot;coming soon&quot;.Still, exciting times, can&#x27;t wait to get my hands on it!

Well, the a fine tuned version of the Pro model now powers Bard - which is free; so it’s probably quite cheap (to Google at least).

I’m most curious about the efficiency of the model in terms of computer needed per query.

Exciting to see more progress and options in this space. My personal opinion is that more competition in this space is better than one single player capturing the entire market.

It seems to be. Bard is only using the G-Pro model, not the Ultra, which is what all the benchmarks they&#x27;re touting are showing. If I had to guess, the best you could hope for is exactly what you&#x27;re describing.

I don&#x27;t understand how anyone can see a delayed EU launch as anything other than a red flag. It&#x27;s basically screaming &quot;we didn&#x27;t care about privacy and data protection when designing this&quot;.

It depends on your region. In general these things take some time (hours) to go live globally to all enabled regions, and are done carefully. If you come back tomorrow or in a few days it&#x27;s more likely to have reached you, assuming you&#x27;re in an eligible region.It&#x27;s probably best to wait until the UI actually tells you Bard has been updated to Gemini Pro. Previous Bard updates have had UI announcements so I&#x27;d guess (but don&#x27;t know for sure) that this would have similar.&gt; Bard with Gemini Pro is rolling out today in English for 170 countries&#x2F;territories, with UK and European availability “in the near future.” Initially, Gemini Pro will power text-based prompts, with support for “other modalities coming soon.”<a href="https:&#x2F;&#x2F;9to5google.com&#x2F;2023&#x2F;12&#x2F;06&#x2F;google-gemini-1-0&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;9to5google.com&#x2F;2023&#x2F;12&#x2F;06&#x2F;google-gemini-1-0&#x2F;</a>

Is it live already at bard.google.com? Just tried it and still useless compared to GPT 3.5.

You seem to be saying the same thing- Googles best work is in the past, their current offerings are underwhelming, even if foundational to the progress of others.

In fairness, the performance&#x2F;size ratio for models like BERT still gives GPT-3&#x2F;4 and even Llama a run for it&#x27;s money. Their tech isn&#x27;t as product-ized as OpenAI&#x27;s, but Tensorflow and it&#x27;s ilk have been an essential part of driving actual AI adoption. The people I know in the robotics and manufacturing industries are forever grateful for the out-front work Google did to get the ball rolling.

Large backers and stakeholders are not 25 years old.

It would be interesting to write a LLM query to separate speech details based on target audience: stakeholders, consumers, etc.

&gt; to me this comes off kind of badly, like it&#x27;s trying too hard to emphasize how long they&#x27;ve been doing AIThese lines are for the stakeholders as opposed to consumers. Large backers don&#x27;t want to invest in a company that has to rush to the market to play catch-up, they want a company that can execute on long-term goals. Re-assuring them that this is a long-term goal is important for $GOOG.

My general conclusion: Gemini Ultra &gt; GPT-4 &gt; Gemini ProSee Table 2 and Table 7 <a href="https:&#x2F;&#x2F;storage.googleapis.com&#x2F;deepmind-media&#x2F;gemini&#x2F;gemini_1_report.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;storage.googleapis.com&#x2F;deepmind-media&#x2F;gemini&#x2F;gemini_...</a> (I think they&#x27;re comparing against original GPT-4 rather than GPT-4-Turbo, but it&#x27;s not entirely clear)What they&#x27;ve released today: Gemini Pro is in Bard today. Gemini Pro will be coming to API soon (Dec 13?). Gemini Ultra will be available via Bard and API &quot;early next year&quot;Therefore, as of Dec 6 2023:SOTA API = GPT-4, still.SOTA Chat assistant = ChatGPT Plus, still, for everything except video, where Bard has capabilities . ChatGPT plus is closely followed by Claude. (But, I tried asking Bard a question about a youtube video today, and it told me &quot;I&#x27;m sorry, but I&#x27;m unable to access this YouTube content. This is possible for a number of reasons, but the most common are: the content isn&#x27;t a valid YouTube link, potentially unsafe content, or the content does not have a captions file that I can read.&quot;)SOTA API after Gemini Ultra is out in ~Q1 2024 = Gemini Ultra, if OpenAI&#x2F;Anthropic haven&#x27;t released a new model by thenSOTA Chat assistant after Bard Advanced is out in ~Q1 2024 = Bard Advanced, probably, assuming that OpenAI&#x2F;Anthropic haven&#x27;t released new models by then

Watching these videos made me remember this cool demo Google did years ago where their earpods would auto translate in realtime a conversation between two people talking different languages. Turned out to be demo vaporware. Will this be the same thing?

Wait so it doesn&#x27;t exist yet? Thanks for watching 45 minutes of video to figure that out for me. Why am I wasting my time reading this thread?Somebody please wake me up when I can talk to the thing by typing and dropping files into a chat box.

When I watch any of these videos, all the related videos on my right sidebar are from Google, 16 of which were uploaded at the same time as the one I&#x27;m watching.I&#x27;ve never seen the entire sidebar filled with the videos of a single channel before.

To add to my comment above: Google DeepMind put out 16 videos about Gemini today, the total watch time at 1x speed is about 45 mins. I&#x27;ve now watched them all (at &gt;1x speed).In my opinion, the best ones are:* <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI</a> - variety of video&#x2F;sight capabilities* <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=JPwU1FNhMOA" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=JPwU1FNhMOA</a> - understanding direction of light and plants* <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=D64QD7Swr3s" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=D64QD7Swr3s</a> - multimodal understanding of audio* <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=v5tRc_5-8G4" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=v5tRc_5-8G4</a> - helping a user with complex requests and showing some of the &#x27;thinking&#x27; it is doing about what context it does&#x2F;doesn&#x27;t have* <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=sPiOP_CB54A" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=sPiOP_CB54A</a> - assessing the relevance of scientific papers and then extracting data from the papersMy current context: API user of OpenAI, regular user of ChatGPT Plus (GPT-4-Turbo, Dall E 3, and GPT-4V), occasional user of Claude Pro (much less since GPT-4-Turbo with longer context length), paying user of Midjourney.Gemini Pro is available starting today in Bard. It&#x27;s not clear to me how many of the super impressive results are from Ultra vs Pro.Overall conclusion: Gemini Ultra looks very impressive. But - the timing is disappointing: Gemini Ultra looks like it won&#x27;t be widely available until ~Feb&#x2F;March 2024, or possibly later.&gt; As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for early experimentation and feedback before rolling it out to developers and enterprise customers early next year.&gt; Early next year, we’ll also launch Bard Advanced, a new, cutting-edge AI experience that gives you access to our best models and capabilities, starting with Gemini Ultra.I hope that there will be a product available sooner than that without a crazy waitlist for both Bard Advanced, and Gemini Ultra API. Also fingers crossed that they have good data privacy for API usage, like OpenAI does (i.e. data isn&#x27;t used to train their models when it&#x27;s via API&#x2F;playground requests).

Indeed, I would think the core search product as another example of ai&#x2F;ml...

I just want to underscore that. DeepMind&#x27;s research output within the last month is staggering:2023-11-14: GraphCast, word leading weather prediction model, published in Science2023-11-15: Student of Games: unified learning algorithm, major algorithmic breath-through, published in Science2023-11-16: Music generation model, seemingly SOTA2023-11-29: GNoME model for material discovery, published in Nature2023-12-06: Gemini, the most advanced LLM according to own benchmarks

It&#x27;s worth remembering that AI is more than LLMs. DeepMind is still doing big stuff: <a href="https:&#x2F;&#x2F;deepmind.google&#x2F;discover&#x2F;blog&#x2F;millions-of-new-materials-discovered-with-deep-learning&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;deepmind.google&#x2F;discover&#x2F;blog&#x2F;millions-of-new-materi...</a>

they have to try something, otherwise it looks like they&#x27;ve been completely destroyed by a company of 1000 people

Unless you are OpenAI, the company, I doubt OP implied it was aimed at you. But then I wouldn&#x27;t know as I am much younger than Sundar Pichai and I am not on first name basis with him either ;-)

Sundar studied material science in school and is only slightly older than me. Google is a little over 25 years old. I guarantee you they have not been doing AI since I was a baby.And how many financial people worth reconning with are under 30 years old? Not many.

&gt; &quot;we&#x27;ve been doing this ai stuff since you (other AI companies) were little babies&quot;Well in fairness he has a point, they are starting to look like a legacy tech company.

That was ages ago. In AI even a week feels like a whole year in other fields. And many&#x2F;most of those researchers have fled to startups, so those startups also have a right to brag. But not too much - only immediate access to a model beating GPT4 is worth bragging today (cloud), or getting GPT3.5 quality from a model running on a phone (edge).So it&#x27;s either free-private-gpt3.5 or cloud-better-than-gpt4v. Nothing else matters now. I think we have reached an extreme point of temporal discounting (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Time_preference" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Time_preference</a>).

&gt; show instead of tellThey showed AlphaGo, they showed Transformers.Pretty good track record.

What an ugly statement. DeepMind has been very open with their research since the beginning because their objective was much more on making breakthroughs with moonshot projects than near term profit.

No surprises here.Google DeepMind squandered their lead in AI so much that they now have to have “Google” prepended to their name to show that adults are now in charge.

&gt; only now is it really clearly &quot;adding shareholder value&quot;.AI has been adding a huge proportion of the shareholder value at Google for many years. The fact that their inference systems are internal and not user products might have hidden this from you.

I also find that tone a bit annoying but I&#x27;m OK with it because it highlights how these types of bets, without an immediate benefit, can pay off very well in the long term, even for huge companies like Google. AI, as we currently know it, wasn&#x27;t really a &quot;thing&quot; when Google started with it and the payoff wasn&#x27;t clear. They&#x27;ve long had to defend their use of their own money for big R&amp;D bets like this and only now is it really clearly &quot;adding shareholder value&quot;.Yes, I know it was a field of interest and research long before Google invested, but the fact remains that they _did_ invest deeply in it very early on for a very long time before we got to this point.Their continued investment has helped push the industry forward, for better or worse. In light of this context, I&#x27;m ok with them taking a small victory lap and saying &quot;we&#x27;ve been here, I told you it was important&quot;.

No this is not correct. Arguably OpenAI invented LLMs with GPT3 and the preceding scaling laws paper. I worked on LAMDA, it came after GPT4 and was not as capable. Google did invent the transformer, but all the authors of the paper have left since.

Didn&#x27;t Google invent LLMs and didn&#x27;t Google have an internal LLm with similar capabilities long before openai released the gpts? Remember when that guy got fired for making a claim it was conscious ?The look isn&#x27;t good. But it&#x27;s not dishonest.

That statement isn&#x27;t really directed at the people who care about the scientific or tech-focused capabilities. I&#x27;d argue the majority of those folks interested in those things already know about DeepMind.This statement is for the mass market MBA-types. More specifically, middle managers and dinosaur executives who barely comprehend what generative AI is, and value perceived stability and brand recognition over bleeding edge, for better or worse.I think the sad truth is an enormous chunk of paying customers, at least for the &quot;enterprise&quot; accounts, will be generating marketing copy and similar &quot;biz dev&quot; use cases.

Oh it&#x27;s good they working on important problems with their ai. Its just openai was working on my&#x2F;our problems (or providing tools to do so) and that&#x27;s why people are more excited about them.
Not because of cultural differences. If you are more into weather forecasting, yeah it sure may be reasonable to prefer google more.

While you are spot on, I cannot avoid thinking of 1996 or so.On one corner: IBM Deep Blue winning vs Kasparov. A world class giant with huge research experience.On the other corner, Google, a feisty newcomer, 2 years in their life, leveraging the tech to actually make something practical.Is Google the new IBM?

Nobody said he&#x27;s wrong. Just that it&#x27;s a bad look.

&gt; and Google has untouchable resources such as all the books they&#x27;ve scanned (and already won court cases about)<a href="https:&#x2F;&#x2F;www.hathitrust.org&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.hathitrust.org&#x2F;</a> has that corpus, and its evolution, and you can propose to get access to it via collaborating supercomputer access. It grows very rapidly. InternetArchive would also like to chat I expect. I&#x27;ve also asked, and prompt manipulated chatGPT to estimate the total books it is trained with, it&#x27;s a tiny fraction of the corpus, I wonder if it&#x27;s the same with Google?

&gt; They do make OpenAI look like kids in that regard.Nokia and Blackberry had far more phone-making experience than Apple when the iPhone launched.But if you can&#x27;t bring that experience to bear, allowing you to make a better product - then you don&#x27;t have a better product.

&gt; They do make OpenAI look like kids in that regard.It makes Google look like old fart that wasted his life and didn&#x27;t get anywhere and now he&#x27;s bitter about kids running on his lawn.

Damn I totally forgot Google actually has rights over its training set, good point, pretty much everybody else is just bootlegging it.

Great. But school&#x27;s out. It&#x27;s time to build product. Let the rubber hit the road. Put up or shut up, as they say.I&#x27;m not dumb enough to bet against Google. They appear to be losing the race, but they can easily catch up to the lead pack.There&#x27;s a secondary issue that I don&#x27;t like Google, and I want them to lose the race. So that will color my commentary and slow my early adoption of their new products, but unless everyone feels the same, it shouldn&#x27;t have a meaningful effect on the outcome. Although I suppose they do need to clear a higher bar than some unknown AI startup. Expectations are understandably high - as Sundar says, they basically invented this stuff... so where&#x27;s the payoff?

I think Apple (especially under Jobs) had it right that customers don’t really give a shit about how hard or long you’ve worked on a problem or area.

They do not make Openai look like kids. If anything, it looks like they spent more time, but achieved less. GPT-4 is still ahead of anything Google has released.

I thought that Google was based out of Silcon Valley&#x2F;California&#x2F;USA

From afar it seems like the issues around Maven caused Google to pump the brakes on AI at just the wrong moment with respect to ChatGPT and bringing AI to market. I’m guessing all of the tech giants, and OpenAI, are working with various defense departments yet they haven’t had a Maven moment. Or maybe they have and it wasn’t in the middle of the race for all the marbles.

He&#x27;s not wrong. DeepMind spends time solving big scientific &#x2F; large-scale problems such as those in genetics, material science or weather forecasting, and Google has untouchable resources such as all the books they&#x27;ve scanned (and already won court cases about)They do make OpenAI look like kids in that regard. There is far more to technology than public facing goods&#x2F;products.It&#x27;s probably in part due to the cultural differences between London&#x2F;UK&#x2F;Europe and SiliconValley&#x2F;California&#x2F;USA.

Not if the little guy leveraged your inventions&#x2F;research.

I do think that’s a backfire. Telling me how long you’ve been doing something isn’t that impressive if the other guy has been doing it for much less time and is better at it. It’s in fact the opposite.

This dude needs a new speech writer. How about we go further and just state what everyone (other than Wall St) thinks: Google needs a new CEO.One more interested in Google&#x27;s supposed mission (&quot;to organize the world&#x27;s information and make it universally accessible and useful&quot;), than in Google&#x27;s stock price.

&gt;&gt; This dude needs a new speech writer.If only there was some technology that could help &quot;generate&quot; such text.

Dude needs a new job. He&#x27;s been the Steve Balmer of Google, ruining what made them great and running the company into the ground.

&gt; Sundar&#x27;s comments in the main video seem like he&#x27;s trying to communicate &quot;we&#x27;ve been doing this ai stuff since you (other AI companies) were little babies&quot; - to me this comes off kind of badlyReminds me of the Stadia reveal, where the first words out of his mouth were along the lines of &quot;I&#x27;ll admit, I&#x27;m not much of a gamer&quot;This dude needs a new speech writer.

Well, deepmind was doing amazing stuff before OpenAI.AlphaGo, AlphaFold, AlphaStar.They were groundbreaking a long time ago. They just happened to miss the LLM surge.

It&#x27;s such a short sighted approach too because I&#x27;m sure someone will develop a GPT with native advertising and it&#x27;ll be a blockbuster because it&#x27;ll be free to use but also have strong revenue generating potential.

The point is if you have to remind people then you’re doing something wrong. The insight to draw from this is not that everyone else is misinformed about googles abilities (the implication), its that Google has not capitalized on their resources.

Its a conceit but not unjustified, they have been doing &quot;AI&quot; since their inception. And yeah, Sundar&#x27;s term up until recently seems to me to be milking existing products instead of creating new ones, so it is a bit annoying when they act like this was their plan the whole time.Google&#x27;s weakness is on the product side, their research arm puts out incredible stuff as other commenters have pointed out. GPT essentially came out from Google researchers that were impatient with Google&#x27;s reluctance to ship a product that could jeopardize ad revenue on search.

This demo is blowing my mind! It&#x27;s really incredible. Can&#x27;t wait to play around with them.

But their first efforts in BARD were really not great. I&#x27;d just have left the bragging out in terms of how long. OpenAI and others have no doubt sent a big wakeup call to google. For a while it seemed like they had turned to focus an AI &quot;safety&quot; (remembering some big blowups on those teams as well) with papers about how AI might develop negative stereotypes (ie, men commit more violent crime then women?). That seems to have changed - this is very product focused, and I asked it some questions that in many models are screened out for &quot;safety&quot; and it responded which is almost even more surprising (ie. Statistically who commits more violent crime, men or women).

Sundar&#x27;s comments about Google doing AI (really ML) are based more on things that people externally know very little about. Systems like SETI, Sibyl, RePhil, SmartASS. These were all production ML systems that used fairly straightforward and conventional ML combined with innovative distributed computing and large-scale infrastructure to grow Google&#x27;s product usage significantly over the past 20 years.For example here&#x27;s a paper 10 years old now: <a href="https:&#x2F;&#x2F;static.googleusercontent.com&#x2F;media&#x2F;research.google.com&#x2F;en&#x2F;us&#x2F;pubs&#x2F;archive&#x2F;41159.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;static.googleusercontent.com&#x2F;media&#x2F;research.google.c...</a> and another close to 10 years old now: <a href="https:&#x2F;&#x2F;research.google&#x2F;pubs&#x2F;pub43146&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;research.google&#x2F;pubs&#x2F;pub43146&#x2F;</a>
The learning they expose in those papers came from the previous 10 years of operating SmartASS.However, SmartASS and sibyl weren&#x27;t really what external ML people wanted- it was just fairly boring &quot;increase watch time by identifying what videos people wioll click on&quot; and &quot;increase mobile app installs&quot; or &quot;show the ads people are likely to click on&quot;.It really wasn&#x27;t until vincent vanhoucke stuffed a bunch of GPUs into a desktop and demonstrated scalable and dean&#x2F;ng built their cat detector NN that google started being really active in deep learning. That was around 2010-2012.

Google literally invented transformers that are at the core of all current AI&#x2F;LLMs so Sundar&#x27;s comment is very accurate.

One observation: Sundar&#x27;s comments in the main video seem like he&#x27;s trying to communicate &quot;we&#x27;ve been doing this ai stuff since you (other AI companies) were little babies&quot; - to me this comes off kind of badly, like it&#x27;s trying too hard to emphasize how long they&#x27;ve been doing AI (which is a weird look when the currently publicly available SOTA model is made by OpenAI, not Google). A better look would simply be to show instead of tell.In contrast to the main video, this video that is further down the page is really impressive and really does show - the &#x27;which cup is the ball in is particularly cool&#x27;: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=UIZAiXYceBI</a>.Other key info: &quot;Integrate Gemini models into your applications with Google AI Studio and Google Cloud Vertex AI. Available December 13th.&quot; (Unclear if all 3 models are available then, hopefully they are, and hopefully it&#x27;s more like OpenAI with many people getting access, rather than Claude&#x27;s API with few customers getting access)

Gemini AI