Some people like a flashy UI, the modern look is important for them. It&#x27;s ok to have aesthetic preferences, let&#x27;s not pretend we don&#x27;t all have them.In the end, it&#x27;s a niche search engine I&#x27;ve made, the intended audience is the long tail. It just isn&#x27;t for everyone, and if it was for everyone, it probably would be lesser for it.

it&#x27;s too &quot;uncompact&quot;. Font size too big and could use a bit more horizontal space.I find it comfortable to use at 60% zoom level

How so? It&#x27;s intuitive and super fast. I whish there were more websites with such a simple UI.

What makes you find the user interface awful?it is litterally a search website with a text box for a search term and a button to do the search.

Ironic comment, considering that this is a search engine to weed out sites with awful UIs. This gives us exactly what we need in a search UI - no more, no less - in a clean and intuitive way.

Yeah the design could use some work. The search results are not compact - I only see 1 result without scrolling, not counting the related wikipedia link that apparently has no description.I don&#x27;t particularly like that it seems to be a column constrained to 550px width, instead of being responsive and taking advantage of greater widths.to the author of the site, if you&#x27;re not really into design&#x2F;css, take a look at tailwindcss, it makes it fairly easy to produce a minimal amount of css that is responsive.

The page isn&#x27;t prompting for cookie preferences, asking to allow notifications, popping up a mailing list or coupon half way do the page, playing a full page video with sound, or load 97million lines of javascript. I&#x27;d say its pretty much perfect.

Huh. I think it&#x27;s a great UI. What did you not like about it?

I wrote it myself from scratch. I have some metadata in mariadb, but the index is bespoke.A design sketch of the index is that it uses one file with sorted URL IDs, one with IDs of N-grams (i.e. words and word-pairs) referring to ranges in the URL file; as well as a dictionary for relating words to word-IDs; that&#x27;s a GNU Trove hash map I modified to use memory map data instead of direct allocated arrays.So when you search for two words, it translates them into IDs using the special hash map, goes to the words file and finds the least common of the words; starts with that.Then it goes to the words file and looks up the URL range of the first word.Then it goes to the words file and looks up the URL range of the second word.Then it goes through the less common word&#x27;s range and does a binary search for each of those in the range of the more common word.Then it grabs the first N results, and translates them into URLs (through mariadb); and that&#x27;s your search result.I&#x27;m skipping over a few steps, but that&#x27;s the very crudest of outlines.

Which software do you use to index the sites?

Hi,Interesting idea. Definitely see an overlap with eReader markets and looking at text only contents.How does it work?It ignores pages on which it detects frameworks for ui and ads or any javascript code at all?

Javascript tags are penalized the hardest, but it also takes into consideration density of text per HTML. There&#x27;s also some adjustments based on text length, which words occur in the page, etc.

I love this, and I love (many of) the results so far! What I can&#x27;t find on the site is detail about what &quot;too many modern web design features&quot; means. Is it just penalizing sites with tons of JavaScript?

About 21 million, and I&#x27;m crawling myself.Classification is really hard. I&#x27;m struggling with it myself, as a lot of like privacy policies and change logs turns out to share the shape of a page of text.I&#x27;m thinking of experimenting with ML classifiers, as I do have reasonably good ways of extracting custom datasets. Finding change logs and privacy policies is easy, excluding them is hard.

Very cool project! How many websites do you have in your index? And how did you go about building it?I&#x27;ve been working on an engine for personal websites, currently trying to build a classifier to extract them from commoncrawl, if you have any general tips on that kind of project they&#x27;d be very welcome.

If Google starts showing interesting text-heavy links instead of vapid listicles and storefronts, I have accomplished everything I ever could dream of.

Love it, kudos! This is great for developers and others who Just Need Answers and not shopping or entertainment.If you&#x27;re looking for feedback, both from a UI design and utility standpoint, you might consider &quot;inlining&quot; results from selected sites, e.g. Wikipedia, stacked change, etc. Having worked on search for a long time, inlining (onebox etc) is a big reason users choose Google, and that channelers fail to get traction. If you&#x27;re Serious(tm), dog into the publisher structure formats and format those, create a test suite, etc.A word of caution: if this takes off, as a business it&#x27;s vulnerable to Google shifting its algorithms slightly to identify the segment of users+queries who prefer these results and give the same results to those queries.Hope this helps!

It&#x27;s pretty much all bespoke.I use external libraries for parsing HTML (JSoup) and robots.txt; but that&#x27;s about it.

Nice, what are you using to crawl the web?

Great work. Working on an alternative search engine too. Take a look at my profile.

Hmm, which OS? I only have a single Android phone so I&#x27;ve only fixed the CSS for that.

Fennec F-Droid on Android 11 has some rendering issues.

Pretty neat!!!You may already be aware of this, but the page doesn&#x27;t seem to be formatted correctly on mobile. The content shows in a single thin column in the middle.

I would also love to add some sites which I were missing...Unfortunately this doesn&#x27;t seem to be a feature which new search engines are focusing on - Brave Search also misses that feature...

This is awesome. I&#x27;ve been looking for a long time for a search engine that basically takes everything Google does and does the opposite. Thank you for doing this, I will definitely be bookmarking it.Is there a way to suggest or add sites? I went looking for woodgears.ca and only got one result. I also think my personal blog would be a good candidate for being indexed here but I couldn&#x27;t find any results for it.

I think you need to approach it more like grep than google. It&#x27;s a forgotten art, dealing with this type of dumb search engine.Like if you search for &quot;How do I make a steak&quot;, you aren&#x27;t going to get very good results. But a better query is &quot;Steak Recipe&quot;, as that is at least a conceivable H1-tag.

It&#x27;s very rare that I see a project on HN I can see myself using. This is one. Like others have said, the results can be a little rough. But they&#x27;re rough in a way I think is much more manageable than the idiosynchrosies of more &#x27;clever&#x27; search engines.

Seconded, I’d like to incorporate it into a project of mine.

is there a json endpoint ? I&#x27;d love to make an emacs bridge :)

Not OP but crawling is easy if you don&#x27;t try scanning 5+ pages a second - almost all rate limiting&#x2F;heuristic based &#x27;keep server costs low&#x27; engines, including Cloudflare, don&#x27;t care if you request every page, but will take action if you do something like burst every page and take up just as many server resources as a hundred concurrent users.Now, that is assuming you aren&#x27;t on some VPS provider. If you&#x27;re going to crawl, you&#x27;ll have the best chance when you use your own IPs on your own ASN, with DNS and reverse DNS set up correctly. This makes it so the IP reputation systems can detect you as a crawler but not one that hammers every site it visits.Also, I imagine that, for a search engine like this, it doesn&#x27;t expect content to change much anyways - so it can take its time crawling every site only once every month or two, instead of the multiple times a week (or day) search engines like Google have to for the constantly-updated content being churned out.

How are you doing the crawling without getting blocking? -- the hardest part.

This is absolutely wonderful. I am LOVING the results I&#x27;m getting back from it: the sort of content-rich sites that have become nigh unreachable using traditional search engines. Thank you for building this!

those three pages sound pretty interesting, I don&#x27;t see this as a problem

I don’t want my search engine to somehow try to judge the believability of the results. I’d like to be the judge of that myself.

I love this idea, and admire the work you put into it. I&#x27;m a fan of long reads and historical non-fiction, and Google&#x27;s results are truly garbage.I have a criticism that I think may pertain to the ranking methodology. I searched for &quot;discovery of Australia&quot;. Among the top results were:* A site claiming that the biblical flood was caused by Earth colliding with a comet (with several other pages from that site also making the top search results with other wild claims, e.g. that the Egyptians discovered Arizona);* Another site claiming the first inhabitants of Australia were a lost tribe of Israel;* A third site claiming that Australia was discovered and founded by members of a secret society of Rosicrucians who had infiltrated the Dutch East India Company and planned to build an Australian utopia...These were all pages heavy with HTML4 tags and virtually devoid of Javascript, the kinds of pages you&#x27;d frequently see in the late 1990s from people who had built their own static websites in a text editor, or exported HTML from MS Word. At that time, there were millions of those sites with people paying for their own unique domain names, and so the proportion of them that were home to wild-eyed conspiracy theories was relatively small. What I think has happened is that kooks continued to keep these sites up - to the point where it&#x27;s almost a visual trope now to see a red &lt;h1&gt; tag in Times New Roman and think, uh oh, I&#x27;ve stumbled on an &quot;ancient aliens&quot; site. Whereas scholars and journals offering higher quality information have moved to more modern platforms that rely more heavily on modern browsers - with or without their own domain names. So as a result what seemed to surface here were the fragments of the old web that remain live - possibly because people living in cabins in Montana forget to cancel their web hosting, or because the nature of old-school conspiracy theorists is to just keep packing their old sites with walls of text surrounded by &lt;p&gt; tags.Arguably, this seems to rank the way Google&#x27;s engine used to, since it couldn&#x27;t run JS and they wanted to punish sites that used code to change markup at render time. At least, when I used to have to do onsite SEO work, it was always about simple tag hierarchies.I wonder whether there isn&#x27;t some better metric of validity and information quality than what markup is used. Some of the sites that surfaced further down could be considered interesting and valuable resources. I think not punishing simple wall-of-text content is a good thing. But to punish more complicated layouts may have the perverse effect of downranking higher-quality sources of information - i.e. people and organizations who can afford to build a decent website, or who care to migrate to a modern blogging platform.

I searched Warcraft and got a gold selling&#x2F; level  boosting site. Some things never change :)

&gt; How are you able to keep the site running after HN kiss of death?I originally targeted a Raspberry Pi4-cluster. It was only able to deal with about 200k pages at that stage, but it did shape the design in a way that makes very thrifty use of the available hardware.My day job is also developing this sort of highly performance java applications, I guess it helps.&gt; What is your stack, elastic search or something simper?It&#x27;s a custom index engine I built for this. I do use mariadb for some ancillary data and to support the crawler, but it&#x27;s only doing trivial queries.&gt; How did you crawl so many websites for a project this size?It&#x27;s not that hard. Like it seems like it would be, and there certainly is an insane number of edge cases, but if you just keep tinkering you can easily crawl dozens of pages per second even on modest hardware (of course distributed across different domains).&gt; Did you use any APIs like duck duck go or data from other search engines?Nope, it&#x27;s all me.&gt; Are you still incorporating something like PageRank to ensure good results are prioritized or is it just the text-based-ness factor?I&#x27;m using a somewhat convoluted algorithm that takes into consideration the text-based-ness of the page, but also how many incoming links the domain has, but it&#x27;s a weighted value that factors in the text-based-ness of the origin domains.It would be interesting to try a page rank-style approach, but my thinking is that because it&#x27;s the algorithm, it&#x27;s also the algorithm everyone is trying to game.

Awesome project! How are you able to keep the site running after HN kiss of death? What is your stack, elastic search or something simper? How did you crawl so many websites for a project this size? Did you use any APIs like duck duck go or data from other search engines? Are you still incorporating something like PageRank to ensure good results are prioritized or is it just the text-based-ness factor?

This has amazing potential. I&#x27;d encourage you to form a non-profit, turn this into something that can last as an organization without becoming what you&#x27;re trying to avoid becoming. This is a good enough start that I bet you could raise a sizeable startup fund very soon from a combination of crowdfunding and foundation grants—I bet the Sloan Foundation would love this!

Love the idea. A little feedback: layout needs tweaking for mobile. FWIW: I&#x27;m on mobile Firefox for Android.

Thank you so much for creating such a useful search engine!Is there any way that you can get an HTTP certificate?I use an old iPhone 4S, and most of the modern web is inaccessible due to TLS. Hacker News and mbasic.facebook are two of the last sites I can use.Usually text-based sites are more accessible, so this could be really useful to help me continue using my antique devices!

I can&#x27;t see the letters in the disturbing white search box, I&#x27;m on duckduckgo, brave, monocles, jquarks, smartcookieweb on Android.

I&#x27;ve mostly added these converters and frills because I&#x27;m trying to eat my own dogfood, and google is deeply engrained as my go-to source for unit conversions and back-of-envelope calculations.Don&#x27;t worry, this stuff is easy, and doesn&#x27;t even remotely take away from the work on the harder problems.

Really like it, and love that you&#x27;ve done this yourself.I&#x27;d prefer if it does just one thing and does that really well. Don&#x27;t waste your time on calculator and conversion functions, or pseudo-natural language queries. There are plenty of good calculator &amp; converter tools and websites, but we all need a really good search engine. I think you&#x27;d be better looking at handling synonyms and plurals.Thanks.

It&#x27;s custom java code, for the most part. I&#x27;m using mariadb for some ancillary information, but the index and the crawler and everything is written from scratch.It&#x27;s hosted on my consumer-equipment server (Ryzen 3900x, 128Gb ram, Optane 900p+a few IronWolf drives), bare bones on Debian.

I continue to want a communal set of 50-100 million urls and data that are &quot;good&quot; (i.e. for any value of good and more complete than common crawl) that are accessible enough to be easy to work with that can be used to experiment with different web search tech. There are enough separate problems to tackle in web search that breaking it down would maybe move the needle. We have lots of kaggle competitions about ranking, but using closed data. What other types of kaggle projects would help web search?What can we do to foster a sustainable bazaar of projects to make it easier to build web search engines?

This is really cool! A spacer between the links would help my old eyes; I keep getting lost in which link goes with which description. :-)

Nice project, but have you heard of FrogFind? it also presents lightweight search results.

Yeah so this is my project. It&#x27;s very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)

The &quot;surprise me...&quot; button is adequately labelled.

See also: <a href="https:&#x2F;&#x2F;wiby.me&#x2F;" rel="nofollow">https:&#x2F;&#x2F;wiby.me&#x2F;</a>

now Google try immitate a document system on your computer, usually I rely on Google know what I need:(

If Google isn&#x27;t useful then nobody will use it.The search engine and the ads are tightly coupled. A better search engine means it can predict with more accuracy what you are looking for and can serve you an even more targeted ad that increases the chance you&#x27;ll click.

I wouldn&#x27;t say the existence of this page proves your prediction right (as it&#x27;s not dethroning Google anytime soon).It&#x27;s easy to forget that the goal of Google isn&#x27;t to provide a useful search engine (at least not anymore), but the search engine is a by product of them wanting to show ads.

You can limit the search query per website in DDG (and probably in others)Example: `rust slow compilation site:stackoverflow.com`

...especially if you could group online resources by category (e.g. software eng, cooking, ...)

As dev I would love search engine which would only do search to stackoverflow github issues, documentation etc.

&gt; iPad dethroned Windows PCs for average home useris this true? in the US, perhaps?
because in south america it couldn&#x27;t me more far away from truth - didn&#x27;t happen at all

Wouldn&#x27;t the dethroner of Google be some new technology which is not a search engine like Google but better at solving the original task of finding information on how to solve problems?Just like how iPad dethroned Windows PCs for average home user but not Mac because Windows had the monopoly and then an innovation destroyed MS in this space and not a competitor.I don&#x27;t think Google dethrones Yahoo and AltaVista scenario will occur again.

I agree, except it&#x27;ll optionally accept the ID of your node in a web of trust, and it&#x27;ll use a page rank customized for you.Or you can put in two ID&#x27;s and have it find sources that both parties trust.

As everything in life flows in cycle, I predict the search engine that will de-throne Google will be like Google when it started - a simple variation of page rank.No smarts, no bubble, no signals decided by over fitting to a biased engineer preference.

How did you go about seeding your web crawler with URLs to crawl?

This is unbelievably impressive on a technical and ambition level for a solo, self-hosted hardware project. Kudos.

Amazing!I have only one recommendation that might make the search a bit more relevant, e.g when searching for &#x27;linux locking&#x27; or &#x27;kernel locking&#x27; kind of things.Try to upsort things that match near the top of the content, like the top of the man page vs middle vs bottom.One easy way to do it without having to store the positions, is to index the ngrams with max(sqrt,8) of their line number, this will cover first 64 lines, you can also use log() or just decide ad hock, top, middle, bottom of the document, so you can use only 3 values.e.g. <a href="https:&#x2F;&#x2F;www.kernel.org&#x2F;doc&#x2F;html&#x2F;v5.0&#x2F;kernel-hacking&#x2F;locking.html" rel="nofollow">https:&#x2F;&#x2F;www.kernel.org&#x2F;doc&#x2F;html&#x2F;v5.0&#x2F;kernel-hacking&#x2F;locking....</a> would do unreliable_1 guide_1 locking_1 ... then at line 4 kernel_2 locking_2 ... after line 50 ... then_7 ... and after that everything will be _8.then just make the query &quot;kernel locking&quot; to &quot;dismax(kernel_1 OR kernel_2 OR kernel_3...) AND dismax(locking_1 OR locking_2 ...) with some tiebreaker of 0.1 or so, you can also say &quot;i want to upsort things on the same line, or few lines apart&quot; by modifying the query a bit.It works really well and costs very little in terms of space, i tried it at <a href="https:&#x2F;&#x2F;github.com&#x2F;jackdoe&#x2F;zr" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jackdoe&#x2F;zr</a> while searching all of stackoverfow&#x2F;man pages and etc and was pretty surprised by the result.This approach is a bit cheaper than storing the positions because positions are (lets say) 4 bytes per term per doc, while this approach has fixed uppre bound cost of 8*4 per document (assuming 4 byte document ids) plus some amortized cost for the terms

&gt; It also focuses on websites in English, Swedish and Latin and tries to identify and ignore the restWhen I search for Japanese terms, it &quot;says &lt;query&gt; needs to be a word&quot;, which wasn&#x27;t the best error message. Maybe the error message should say something like &quot;sorry, your language isn&#x27;t support yet&quot;?

Do you know what proportion of the texty web instructs unknown crawlers to go away (or blocks them)?

Cool, I&#x27;ve been thinking on this topic a bit lately. Crawling is indeed not that hard of a problem. Google could do it 23 years ago. The web is a bit bigger now of course but it&#x27;s not that bad. Those numbers are well within the range of a very modest search cluster (pick your favorite technology; it shouldn&#x27;t be challenging for any of them). 10x or 1000x would not matter a lot for this. Although it would raise your cost a little.The hard problem is indeed separating the good stuff from the bad stuff; or rather labeling the stuff such that you can tell the difference at query time. Page rank was nice back in the day; until people figured out how to game things. And now we have bot farms filling the web with nonsense to drive political agendas, create memes, or to drown out criticism. Page rank is still a useful ranking signal; just not by it self.The one thing no search engine has yet figured out is reputability of sources. Content isn&#x27;t anonymous mostly. It&#x27;s produced and consumed by people. And those people have reputations. Bot content is bad because it comes from sources without a credible reputation. Reputations are built over time and people value having them. What if we could value people&#x27;s appreciation relative to their reputability? That could filter out a lot of nonsense. A simple like button + a flag button combined with verified domain ownership (ssl certificates) could do the trick. You like a lot of content that other people disliked, your reputation goes down the drain. If you produce a lot of content that people like, your reputation goes up. If a lot of reputable people flag your content, your reputation tanks.The hard part is keeping the system fair and balanced. And reputability is of course a subjective notion and there is a danger of creating recommendation bubbles, politicizing certain topics, or even creating alternative reality type bubbles. It&#x27;s basically what&#x27;s happening. But it&#x27;s mostly powered by search engines and social media that actually completely ignore reputability.

I do indeed index the web myself. Not the entire web, just a subset of it. The crawler quickly loses interest in javascript:y websites and only indexes at depth those websites that are simple. It also focuses on websites in English, Swedish and Latin and tries to identify and ignore the rest (best-effort).You&#x27;d be surprised how much you can do with modern hardware if you are scrappy. The current index is about 17.7 million URLs. I&#x27;ve gone as far as 50 million and could probably double that if I really wanted to. The difficulty isn&#x27;t having a small enough index, but rather having a relevant enough index, weeding out the link farms and stuff that just take space.I only index N-grams of up to 4 words, carefully chosen to be useful. The search engine, right now, is backed by a 317 Gb reverse index and a 5.2 Gb dictionary.

Where does the data come from? Do you index the whole web yourself? I see it totally impossible for a personal project. I&#x27;m very curious about that.

What we need now is a search engine that weeds out sites that have been SEO optimized for keyword density.I’m tired of searching for “generic keyword” and getting a page with an extremely low signal to noise ratio written like this:“Many people search for generic keyword. That is why you can find all about generic keyword here. In fact we specialize in generic keyword and slight alterations of generic keyword.”It’s like Google stopped caring that people were gaming it.

I love this! I&#x27;ve been searching random words with no aim in particular and keep finding lots of interesting tiny personal webpages.  It feels like the old web

Aha, I was going to ask how you were coping with CDNs like Cloudflare blocking bots. It&#x27;s sad we&#x27;ve got to this point where basically only the established search engines are grandfathered in to be able to crawl sites.

I wonder if google had to plead the same way or if already-big players are given a free advantage.

Yeah it&#x27;s hosted by cloudflare. I&#x27;m currently IP-blocking them, as because they keep prompting my crawler with a captcha, presumably because it&#x27;s made millions of requests from their CDN.Some rigmarole getting recognized as a good bot by the CDNs. I&#x27;ve submitted a request fairly recently, but haven&#x27;t heard back from them yet.Like I would like to be on good terms with them, and other websites that block small independent crawlers.I can&#x27;t blame them though, there&#x27;s a lot of bad bots out there. But I&#x27;m doing my best not be part of the problem.

&gt; I&#x27;m developing a text-heavy siteI looked at the source for your site&#x27;s front page. That&#x27;s not text-heavy; that&#x27;s markup-heavy. I didn&#x27;t bother looking at the rest of the pages because it appears to be yet another crypto market site.

I&#x27;m developing a text-heavy site and philosophically I&#x27;m trying to view documents as just that... documents [1].But I don&#x27;t get good results for &quot;rug pull&quot;.- 1 <a href="https:&#x2F;&#x2F;rugpullindex.com" rel="nofollow">https:&#x2F;&#x2F;rugpullindex.com</a>

Very interesting because of the interesting results from random websites. It&#x27;s a great discovery tool.Now hoping for search engine that favors text-heavy sites and punishes paywalls

This is really cool, it filters out all fluff.It&#x27;s not always taking me to totally relevant sites but the results contain my favourite type of content.Full of writing and pure html - usually the hallmark of someone who knows what they are doing, wants to communicate but doesn&#x27;t want to waste their time.

The real test is in now, index server is reconstructing its index. It does this every 6 hours if there is new pages. Takes half an hour or so usually.It&#x27;s supposed to be able to handle searches at the same time, but jeepers, it&#x27;s gonna have to chew through nearly 400 Gb of data while dealing with over 1 request per second.

Well,... yet. Load average is at 1.2, not that bad. But the services are getting a solid workout.

That crossed my mind too, considering vanilla webpages sometimes struggle with a top page HN thread, never mind a search engine backend.

Fast and doesn&#x27;t crash when on the front page of Hacker News!

Nice! Typing my name in, gets my own site back as 3 of the top 5 results. I suddenly feel important ;)

I like the idea. Tangentially, I wonder how one would find the right &#x27;penalty&#x27; for more updated sites?

I think I like this method the most, honestly. There&#x27;s something to be said for minimalist web design, but there&#x27;s also something to be said for everything we&#x27;ve learned since the 90s.

Million Short [1] offers an option to omit results from popular domains. It&#x27;s a different approach from what you describe, but I think the goal is similar.[1] <a href="https:&#x2F;&#x2F;millionshort.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;millionshort.com&#x2F;</a>

It&#x27;s hard, though. I simultaneously want something far from my tastes, but I don&#x27;t want to see Plandemic-style Ivermectin material, or Focus On The Family-style material. I want things that will push me out of my comfort zone sometimes, but it turns out I really don&#x27;t want the thing furthest from my tastes; I want things marginally adjacent. I want them close enough to feel familiarity, but far enough that it challenges my worldview.I don&#x27;t think a recommendation engine can do that.

Doing that takes real work and curiosity. I&#x27;m afraid an algorithm will never be able to do it, particularly if you&#x27;re into niche stuff. For instance I enjoy a lot a Japanese band called The Boredoms - but few people like it, and there&#x27;s only 2 of their albums available in spotify.

Similarly, I wish there was a recommendation engine (for web, music, movies, whatever) that can show you what is the furthest away from your existing tastes. I&#x27;ve learned to re-create my Spotify account once every 6 months or so, as their recommendation engine becomes a boring machine after using it daily for some months.I&#x27;d love to discover new content that is different from what I read&#x2F;watch&#x2F;listen to now, but it&#x27;s really hard to know about genres you don&#x27;t know about.

I had this problem recently trying to fix an Atari. There&#x27;s a guy out there who has ton&#x27;s of guides on doing video out mods but newer guide references the older. However googling the OG guide didn&#x27;t find it so I manually scoured his old web page.

That was a great read, thanks for sharing.

Try this <a href="https:&#x2F;&#x2F;wiby.me&#x2F;" rel="nofollow">https:&#x2F;&#x2F;wiby.me&#x2F;</a>Clicking &#x27;Surprise me&#x27; gave me an interesting article from 1994 <a href="http:&#x2F;&#x2F;milk.com&#x2F;wall-o-shame&#x2F;bucket.html" rel="nofollow">http:&#x2F;&#x2F;milk.com&#x2F;wall-o-shame&#x2F;bucket.html</a>

I like it.Coincidentally, the other day I was daydreaming about a search engine that favors sites that are updated less frequently. The thought being, the kinds of labors of love that characterized the 1990s Web that I still sometimes miss are still out there, it&#x27;s just harder to find them amidst the flood of SEO dreck. So perhaps they could be made discoverable again with the help of a contrarian search engine that specifically looks for the kinds of things that Google and Bing don&#x27;t like to see.

Definitely low signal to noise. Looking at you recipe websites and cooking blogs.

It really depends on what you search for. A major drawback is that there needs to be text-heavy sites to find, in order for the search engine to find them.Compare for example the results for &quot;Duke Nukem 3D&quot; with those for &quot;Cyberpunk 2077&quot;.

I tried a few queries and got extremely irrelevant results

There’s certain grey literature that’s not captured in university library federated searches nor easily found with mainstream search engines.

Perhaps a counter example, something that is interesting. Anecdotally. This, of all things, is the top result in my search: <a href="https:&#x2F;&#x2F;tft.brainiac.com&#x2F;archive&#x2F;0303&#x2F;msg00037.html" rel="nofollow">https:&#x2F;&#x2F;tft.brainiac.com&#x2F;archive&#x2F;0303&#x2F;msg00037.html</a>. Which is strange to me because I don&#x27;t recognize tft.brainiac. I click, it&#x27;s a list of biological relationships among Hymenoptera, including a reference to genus of the wasps I studied, presumably in a biological relationship (host&#x2F;parasite) context. I cataloged every relationship known at one point, so my brain wants to know where this come from, is it something I caught. Then I go look for more context, and find it&#x27;s part of a thread about D&amp;D(?) and hymenoptera, and it&#x27;s epic, and a chunk of my morning is lost figuring out why and how this came to be.

In part, by boring results I mean I instantly recognize the top results, and I know exactly what will be in them, and I know which ones will actually contain potentially interesting new stuff, i.e. _I didn&#x27;t have to search for these, I&#x27;d go their directly_. Then next results are all obscure, and I&#x27;ve already visited them, and&#x2F;or I know they are historical and not something I have to revisit.With this engine with at least 1&#x2F;2 the links (to be fair there were &lt; 20) I didn&#x27;t recognize the URL at all, and it was clear in the text or the URL that there was an interesting bit to check out (i.e. what Google should have also returned after they barfed out the things I don&#x27;t need to know about), but had never succinctly done in my experience.I suppose the magic in this engine would have to be alerting the searcher that they found more of this type of link, as once I visited the 10 or so sites they would fall back into the &quot;been there, done that&quot; link category that Google appends somewhere after the ads and &quot;big&quot; sites, mixed in with a million search term spam sites, etc.

I&#x27;m intrigued by this experiment but I can&#x27;t visualize it. What do you mean by boring results? Would combing through a library (the one with paper books) also produce boring results? What&#x27;s your ideal results?

Not sure, but I remember when Google could find literally anything. Then they started adding a bunch of exceptions and crapped out their quality. I wonder how insanely different results would be to get the older Google Engine from the 2000s search result wise.I now have to play games with Google to find things. I feel like I do less than I used to for some reason.

IMHO, that is a trendy claim in HN with little evidence.

This needs an update but is an easy look see.
<a href="https:&#x2F;&#x2F;www.searchenginemap.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.searchenginemap.com&#x2F;</a>Broad and longer Twitter lists maintained here: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;SearchEngineMap&#x2F;lists" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;SearchEngineMap&#x2F;lists</a>

Mojeek was built in the same spirit (one server living in a house) and has 4.5 bn pages indexed now, and a bunch more servers. A lot of people comment in similar style of it reminding them of an older Internet, or generally less branded results. It&#x27;s definitely an alternative point of view. Disclaimer: I work for them.

&gt; well we know Google isn&#x27;t for search anymoreDo you suggest anything better? As far as I can tell, all the other search engines are either repackaged Bing (ie: DuckDuck), or are just as bad.

Exactly this. A couple results returned reference to obscure now-defunct newsletters and clubs, people that I know were historically important for past researchers, but only because this was my research forcus for so long would I have known this.

The search engine doesn&#x27;t actually do full text search, so maybe your query was too... unique.But do first of all verify that you haven&#x27;t been hacked. There&#x27;s about quarter of a million domains I&#x27;ve flagged that, besides their wordpress content, also host a ton of link spam crap off in some hidden folder. This reflects on the quality rating extremely negatively to the point where you may have not been indexed at all.Secondly, are you behind cloudflare or some other big-name CDN? Because, as I mentioned in another comment, I can&#x27;t crawl their pages without getting captchad until they approve of my humble request to be classified as a good bot.There are some other hosting providers I flat out block on a subnet level because they host a large amount of link farms. This is currently Alibaba, Psychz, eSited, Cloud Yuqu and 1Blu.

Likewise, I co-maintain the only &quot;fan&quot; site on one of my all-time favourite composers&#x2F;performers, and gave the engine a shot with a unique string query. While my text-heavy WP-driven site didn&#x27;t seem to make the cut, the results were highly relevant in that they were links to former band members and collaborators - a couple of which I didn&#x27;t realize existed. That being said, there were a few sites (including my own) I expected to be returned, but no dice. Still, a fascinating experiment that many at HN have been clamouring for.

But what’s their UVP? I’d say quick and relevant search results. And that seems to be constantly degrading.

&gt; well we know Google isn&#x27;t for search anymore,If you&#x27;re talking about the ads, I would bear in mind Google&#x27;s whole business model is basically online advertising. Search is just the vehicle to deliver those ads; I&#x27;d say Google is pretty good at throwing things back.

Fascinating. I studied an &quot;obscure&quot; group of insects. My go-to search term to test an engine is their family name as it is a rarely used word and I know most (all?) of the major data sources that have accumulated data on it. When Wolfram Alpha added species names, I checked with the name, boring, Duck Duck, boring, Google (well we know Google isn&#x27;t for search anymore, it&#x27;s absolutely horrible) boring, Bing, boring... you get the idea.This was a little different, extremely few results, but a couple of them really made me grin, and all(?) made me curious or raise an eyebrow or reflect on who&#x2F;what might have been the source of the link, or remember some obscure connection from grad-school. So, if anything a crawled list of results worthy of ponder, thanks for this!

It looks very nice anyway, great job! I did try with other queries and results were in general interesting.

Yeah I did some work very recently aimed at improving the relevance a bit. It was a bit too random in the state it was before.  Now it, perhaps, isn&#x27;t random enough anymore.

from About page:&gt; If you search for &quot;Plato&quot;, you might for example end up at the Canterbury Tales. Go looking for the Canterbury Tales, and you may stumble upon Neil Gaiman&#x27;s blog.I know it is just a suggestion, but had to try searching both, with no luck in getting the expected unexpected.

Related question - suppose I want to create a meta search engine for myself, and I want it to be as fast as possible. What are the things I should be optimizing for?

Let a thousand search engines bloom.btw, interesting how many http (as opposed to https) sites show up...

I just looked up my last name and found a World class heavyweight weightlifter named Josef Grafl born in 1872 who has an awesome portrait of him on Wikipedia. Never before have I read about that man.I love this.

You can easily do proper SEO with static site generators too. Even more, static sites can be hosted via GitHub or GitLab Pages, Netlify or CloudFlare and in all cases the speed will outperform Wordpress in almost all cases. Also, you have way more control over the output than with Wordpress.

The website itself seems generated with some kind of kick-ass generator from template files (.gmi?)I feel like I&#x27;m stuck with Wordpress.com because it brings me some traffic (whereas something hand-rolled on nsfspeech or digital ocean or whatever would literally be off the edge of the web), but the structure of that is so cool.

As much as the release names for macOS over the years were marketing gimmicks, it does make it a lot easier to zero in on the correct version when doing these types of searches.

My pet peeve with search results is simply that there are ancient technical results that in many cases are irrelevant. If I am searching for a Window error message, I don&#x27;t want some old forum post from 2001, especially if it didn&#x27;t have any answers!What would be cool would be for people who host old stuff to &quot;archive&quot; it at some point so it doesn&#x27;t appear in normal results, only if you tick &quot;include archives&quot;.

I tried “Error 49” as a search phrase.It’s rudimentary but no IT-related result.

Yeah there was unfortunately a problem with the content-type code recently, it unfortunately categorized some binary data as HTML and tried to process it best-effort. So there&#x27;s some binary soup in the index.The bug has since been fixed, but it won&#x27;t come into effect in a few weeks.

Searched for my initials - got back a bunch of raw binary results (mp4, pdf, img, txz, etc.) which was disconcerting. Although it did find one reference to actual-me which is better than Google manages on the first 4 pages...<a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;n2xro2Y" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;n2xro2Y</a>

It shouldn&#x27;t give you more than three per domain on paper, but I&#x27;ve got issues with deduplication so sometimes you get six :-(Still a lot of work needs to be done.

I&#x27;ve got some results where the same site is has than 70% of the links. It was a very on topic and high quality site, but still, all the results shouldn&#x27;t point to the same place.I think some grouping by site (and capping to only the few most relevant links there) would improve the engine.

I mean a blacklist system like Twitter&#x27;s, where you block a website forever. Pinterest would be the first to go.

It could get tedious depending on how many sites you want to block, but you can add &quot;-site:google.com&quot; to exclude google.com, for instance.

I wish we could configure Google&#x27;s algorithm to our needs, and blacklist websites.

I guess it&#x27;s a matter of taste. I can barely read anything on regular wikipedia because the inline links disrupt my flow.

Wikpedia links point to &lt;<a href="https:&#x2F;&#x2F;encyclopedia.marginalia.nu&#x2F;" rel="nofollow">https:&#x2F;&#x2F;encyclopedia.marginalia.nu&#x2F;</a>&gt; instead, which to my eyes is less readable. The justified text, done with CSS, instead of the LaTeX algorithm, looks wild. The font used for quotations is even worse (very thin).Wikipedia is perfectly usable without JavaScript and it&#x27;s one of the nicest sites out there typography-wise, so I&#x27;d reconsider this redirection.

Tested with the first person to settle on Island: <a href="https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=Ingolf+Arnarson" rel="nofollow">https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=Ingolf+Arnarson</a>and it worked surprisingly well.Anyone else has good examples?

&gt; Don&#x27;t be afraid to scroll down in the search resultsI never knew it was fear that was preventing me from scrolling

Very cool. A person can really appreciate simple web design looking at something like Luke Smith&#x27;s recipe page.So how on earth do you take an idea like this and scale it for both broad web coverage and high traffic? For that matter, just how much &#x27;useful&#x27; text is there on the net?

I do think this is a great tool, I didn’t intend to come off as being critical. I think I had some misdirected frustration from people on HN talking about how the modern web is bad and should be replaced with a text protocol. I aspire to create web apps that push the boundaries of what is possible today, and feel disappointed whenever I encounter people advocating for regression.But I digress. In hindsight, my comment was obnoxious and under-appreciative of the tool being shared, and my rant was only tangentially related.

Is this a criticism? It doesn&#x27;t at all seem touted as a drop-in replacement for the modern search engine.

Ok this is great if all I want to do is read text, but often times that is very much not all I want to do.  The web is much more than text and images these days.  I can appreciate this as long as it’s branded as a search engine for blogs and articles specifically,  as opposed to being touted as a drop-in replacement for the modern search engine.

This is wonderful and stupendous.I’ve often thought that Google could be turned back into a good search engine by simply eliminating the crap and letting the useful sites float to the top of the results.marginalia.nu seems to like my sites, so it must be good!Some results are prefixed with ! or an arrow dingbat. What does that mean?

Kind of reminds me of the past like alta vista and dogpile.

I wonder if there&#x27;s any mileage in an extension of something like uBlock Origin&#x27;s lists of ad networks to block but instead it&#x27;s a list of known content mills and SEO spam factories to remove from search results?

Oh, I dream of a day where there are multiple useful search engines, specialized for different purposes.You&#x27;re doing God&#x27;s work here. Thanks and good luck.

This sort of optimization is why simple recipes are typically found at the end of a rambling pointless blog post now.Still, the best way to break SEO is to have actual competition in the search space. As long as SEO remains focused on Google there is an opportunity for these companies to thrive by evading SEO braindamage.

I don&#x27;t think the existing media-heavy websites are gaming Google to rank higher. It&#x27;s that Google itself prefers media heavy content; they don&#x27;t have to &quot;game&quot; anything.I also think a search engine like this would be quite hard to game. An ML-based classifier trained on thousands of text-heavy and media-heavy screenshots should be quite robust and I think would be very hard to evade, so the &quot;game&quot; will become more about how identify the crawler so you can serve it a high-ranking page while serving crap to the real users, and it seems fairly easily to defeat if the search engine does a second pass using residential proxies and standard browser user agents to detect this behavior (it could also threaten huge penalties like the entire domain being banned for a month to even deter attempts at this).

If there were a wider variety of popular search engines, with different ranking criteria, would sites begin to move away from gaming the system? Surely it would be too hard to game more than one search engine at a time?

&gt; the people who want to game it will appear.So just add human review to the mix, if a site is obviously trying to game the system (listicles, seo spam etc) just drop and ban them from the search index.

There should be some perfect balance where this search engine is N% as popular as Google, where Google soaks up all of the gamifiers, but this search engine is still popular enough to derive revenue and do ML and other search-engine-useful stuff.

Specialization mostly a problem in monocultures.If you almost only plant wheat, you are going to end up with one hell of a pest problem.If you almost only have Windows XP, you are going to have one hell of a virus problem.If you almost only have SearchRank-style search engines (or just the one), you are going to have one hell of a content spam problem.Even though they have some pretty dodgy incentives, I don&#x27;t think google suffers quality problems because they are evil, I think ultimately they suffer because they&#x27;re so dominant. Whatever they do, the spammers adapt almost instantly.A diverse ecosystem on the other hand limits the viability of specialization by its very nature. If one actor is attacked, it shrinks and that reduces the opportunity for attacking it.

As long as few people use it, it will be great. Rest assured that the moment it becomes popular, the people who want to game it will appear.

I checked the results of the same query and they seem fine. Lots of speeches and articles about George Washington the US president. There&#x27;s even his beer recipe.As for the results you linked, it&#x27;s part of the zeitgeist to list other entities sharing the same name. Sure, they could use some subtle changes in ranking, but overall the returned links satisfy my curiosity.

This is not a Google competitor, it&#x27;s a different type of search engine with different goals.&gt; If you are looking for fact, this is almost certainly the wrong tool. If you are looking for serendipity, you&#x27;re on the right track. When was the last time you just stumbled onto something interesting, by the way?

What were you expecting to see for British? There must be millions of pages containing that term. Anyway the first screenful from Google is unadulterated crap, advertising mixed with the usual trivia questions.If you are going top claim something is wide of the mark then you really ought to tell us at least roughly where the mark is.

I did a search for &quot;George Washington&quot;First result after Wikipedia:&quot;Radiophone Transmitter on the U.S.S. George Washington (1920)In 1906, Reginald Fessenden contracted with General Electric to build the first alternator transmitter. G.E. continued to perfect alternator transmitter design, and at the time of this report, the Navy was operating one of G.E.&#x27;s 200 kilowatt alternators
<a href="http:&#x2F;&#x2F;earlyradiohistory.us&#x2F;1919wsh.htm" rel="nofollow">http:&#x2F;&#x2F;earlyradiohistory.us&#x2F;1919wsh.htm</a>
&quot;Another result in the first few:&quot; - VANDERBILT, GEORGE WASHINGTONPH: (800) ###-#233 FX: (#03) 641-5###.
<a href="https:&#x2F;&#x2F;www.ScottWinslow.com&#x2F;manufacturer&#x2F;VANDERBILT_GEORGE_WASHINGTON&#x2F;2182" rel="nofollow">https:&#x2F;&#x2F;www.ScottWinslow.com&#x2F;manufacturer&#x2F;VANDERBILT_GEORGE_...</a>
&quot;And just below that terrible result:&quot;I Looked and I Listened -- George Washington Hill extract (1954)Although the events described in this account are undated, they appear to have occurred in late 1928. I Looked and I Listened, Ben Gross, 1954, pages 104-105: Programs such as these called for the expenditure of larger sums than NBC had anticipated. It be <a href="http:&#x2F;&#x2F;earlyradiohistory.us&#x2F;1954ayl2.htm" rel="nofollow">http:&#x2F;&#x2F;earlyradiohistory.us&#x2F;1954ayl2.htm</a>
&quot;Dramatically worse than Google.---Ok, how about a search for &quot;Rome&quot; then? Surely it&#x27;ll pull some great text results for the city or the ancient empire.First result after Wikipedia:&quot;Home | Rome Daily SentinelReliable Community News for Oneida, Madison and Lewis County
<a href="http:&#x2F;&#x2F;romesentinel.com&#x2F;" rel="nofollow">http:&#x2F;&#x2F;romesentinel.com&#x2F;</a>&quot;The fourth result for searching &quot;Rome&quot;:&quot;Glenn&#x27;s Pens - Stores of NoteGlenn&#x27;s Pens, web site about pens, inks, stores, companies - the pleasure of owning and using a pen of choice. Direcdtory of pen stores in Europe.
<a href="http:&#x2F;&#x2F;www.marcuslink.com&#x2F;pens&#x2F;storesofnote&#x2F;roma.html" rel="nofollow">http:&#x2F;&#x2F;www.marcuslink.com&#x2F;pens&#x2F;storesofnote&#x2F;roma.html</a>&quot;Again, dramatically worse than Google.---Ok, how about if I search for &quot;British&quot;?First result after Wikipedia:&quot;BRITISH MINING DATABASEBritish_Mining_Database
<a href="http:&#x2F;&#x2F;www.users.globalnet.co.uk&#x2F;~lizcolin&#x2F;bmd.htm" rel="nofollow">http:&#x2F;&#x2F;www.users.globalnet.co.uk&#x2F;~lizcolin&#x2F;bmd.htm</a>
&quot;And after that:&quot;British Virgin IslandsMany of these photos were taken on board the Spirit of Massachusetts. The sailing trip was organized by Toto Tours. Images Copyright © Lowell Greenberg Home Up Spring Quail Gardens Forest Home Lake Hodges Cape Falcon Cape Lookout, Oregon Wahkeena
<a href="http:&#x2F;&#x2F;www.earthrenewal.org&#x2F;british_virgin_islands2.htm" rel="nofollow">http:&#x2F;&#x2F;www.earthrenewal.org&#x2F;british_virgin_islands2.htm</a>&quot;Again, far off the mark and dramatically worse than Google.I like the idea of Google having lots of search competition, this isn&#x27;t there yet (and I wouldn&#x27;t expect it to be). I don&#x27;t think overhyping its results does it any favors.

Since it does not use synonyms, it looks like it is unable to answer &quot;how&#x27;s that thing called&quot;-queries.

That query is too long apparently. But if you shorten to &quot;haskell type inference&quot;, I think it delivers on its promise:&gt; If you are looking for fact, this is almost certainly the wrong tool. If you are looking for serendipity, you&#x27;re on the right track. When was the last time you just stumbled onto something interesting, by the way?

It would be nice if we could pipe search engines.

However, when searching for &quot;haskell type inference algorithm&quot; I get completely useless results.

You need to use some old school search techniques and search for “Java overview”

I&#x27;m not sure that&#x27;s a bad thing.

well, they&#x27;re results to java related items...What kind of links where you expecting to find?

I had the exact opposite experience. I searched the site for &quot;java&quot;, got a Wikipedia link first (for the island, not the programming language), and the 2nd result was to a random JEP page, and all the rest of the results were random tidbits about Java (e.g. &quot;XZ compression algorithm in Java). Didn&#x27;t get any high level results pointing to an overview of the language, getting started guides, etc.

The you&#x27;d use a different search engine. Why does everything have to be a Swiss Army knife?

Imagine including the search term &quot;movie&quot;.

I tend to prefer Wikipedia for movies. The exception is actor headshots if I&#x27;m trying to identify someone, which Wikipedia lacks for licensing reasons, but otherwise Wikipedia tends to be better than IMDB for most needs. Wikipedia has an IMDB link on every article anyway.Another need I guess might be reviews, for which RT or MC are better than IMDB: not sure if either of those two will fare better than IMDB in this search engine but again Wiki has links out (in addition to good reception summaries)

?q=imdb.com:fall of the roman empire

Imagine if you were looking for the movie.

It&#x27;s also possible that it&#x27;s the other way around: a certain &quot;common denominator&quot; + algorithms that chase broad engagement = mediocre results.The real trick would be some kind of engine that can aim just above where the user&#x27;s at.

The context matters. I&#x27;d happily read &quot;Top 10&quot; lists on a website if the site itself was dedicated to that one thing. &quot;Top 10 Prog Rock albums&quot;, while a lazy, SEO-bait title, would at least be credible if it were on a music-oriented website.But no, these stories all come from cookie-cutter &quot;new media&quot; blog sites, written by an anonymous content writer who&#x27;s repackaged Wikipedia&#x2F;Discogs info into Buzzfeed-style copy writing designed to get people to &quot;share to Twitter&#x2F;FB&quot;. No passion, no expertise. Just eyeballs at any cost.

what I really want is a true AI to search through all that and figure out the useful truth.  I don&#x27;t know how to do this (and of course whoever writes the AI needs to be unbiased...)

&gt; I genuinely believe terrible results like this are making society dumber.You have to e causality reversed. Google results reflect the fact that society is dumb.

So did Tim Berners-Lee. He was vehemently opposed to people shoehorning images into the WWW, because he didn&#x27;t want it to turn into the equivalent of magazines. Which, I believe, he shared the opinion of them making society dumber.Appropriately enough, I couldn&#x27;t find a good quote to verify that since Google is only giving me newspapers and magazines talking about Sir Tim in the context of current events. I do believe it&#x27;s in his book &quot;Weaving the Web&quot; though.

Good comparison. Reminds me of an analogy I like to make of today&#x27;s web, which is it feels like browsing through a magazine store — full of top 10s, shallow wow-factoids, and baity material. I genuinely believe terrible results like this are making society dumber.

If one could create an metric of ad to content ratio from the js used, I would guess that would be a nice differentiator too.

True. Unfortunately many large corporate websites through which you pay bills, order tickets, etc. are becoming infested with JS widgets and bulky, slow interfaces. These are hard to avoid.

Browsers should be cherry picking the most compelling things that people accomplish with complex code and supporting them as a native feature.
Maybe the Browser Wars aren’t keeping up anymore.

Huh. A weighted algorithm, somewhere between Google and the one linked, where you could subtract from sites by amount of JavaScript might be interesting.

Cool, it appears that the trend towards JS may be causing self-selection -- if a page has a high amount of JS, it is highly unlikely to contain anything of value.

I think this is just because of terms you have searched. In my test-searches Wikipedia has not come up once in first position (i think the highest was 3rd in the list).Here&#x27;s what I&#x27;ve tried with a few variations:
golang generics proposal,
machine learning transformer,
covid hospitalization germany[edit] formatting

The Wikipedia link at the top is always given. It would maybe be good to make it a little clearer that it&#x27;s not one of the true results.

Search engines should be something you pay for. Surely search engine powerusers can afford to pay for such a service. If Google makes $1 per user per month or something, that&#x27;s not too high a bar to get over.

Search engines whose revenue is based on advertising will ultimately be tuned to steer you to the ad foodchain. All the incentives are aligned towards and all the metrics ultimately in service of, profit for advertisers. Not in the 99% of people who can convinced to consume something by ads? Welp, screw you.

You wont believe how Claudius looks today!

&gt;followed by a listicle &quot;8 Reasons Why Rome Fell&quot;but arent you curious about the 7th reason? it will surprise you!

Yeah, Google tends to send a lot of junk back.

Wow I used &quot;personality test&quot; and actually got useful articles about personality theory. I&#x27;ll actually use this!

I&#x27;m curious what you searched for.<a href="https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=harry+potter+fanfiction&amp;profile=default" rel="nofollow">https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=harry+potter+fanfi...</a>This seems to return a pretty decent number of sites relating to that (as well as some sites not relating to that).The search engine isn&#x27;t always great at knowing what a page is about, unfortunately.This seemed to return mostly relevant results<a href="https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=%22harry+potter%22+fanfiction&amp;profile=default" rel="nofollow">https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=%22harry+potter%22...</a>

I tried some queries for Harry Potter fanfictions, and the results were pretty much completely unrelated. There weren’t that many results, either.

Interesting choice of search topic.  Are you trying to make an additional point?

If this search engine ever takes off, the listicle writers will just start optimizing for it too, right?

I think it&#x27;s a case where systems diversity can be an advantage. Much like how most malware was historically written for Windows and could be avoided by using Linux, the low-quality search engine bait is created for Google and can be avoided by using a different style of search engine.

No one mentioned the &quot;bonus&quot; audio in the page source:
<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=7fCifJR6LAY" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=7fCifJR6LAY</a>

Wow, that&#x27;s awesome. Great work!For a simple test, I searched &quot;fall of the roman empire&quot;. In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources.When I search on google, I get wikipedia, followed by a listicle &quot;8 Reasons Why Rome Fell&quot;, then the imdb page for a movie by the same name, and then two Amazon book links, which are totally useless.

Have you given any thought on what you will do if you get a DMCA take down request or a request from a person asking you to remove them from search results?

I&#x27;ve also found that brave search gets much better results than google for some programming related topic, simply by not being targeted by blogspam SEO as much. It&#x27;s refreshing to not have to click through 3 auto generated &quot;articles&quot; but to either a) get the documentation straight away or b) find actually human written blog entries.

I did a quick check using the name of the Scottish village I am originally from (or as I should say &quot;far am fae&quot;) and this produced a much more interesting set of links for me than that produced by Google

I think I want a BBS. Text mode, fixed width font, keyboard-driven menus, no (or very little) bitmapped graphics. I&#x27;ve been thinking about the UIs for a lot of sites that I use to &quot;do things&quot; on the web. E.g. search for flights. Do I need any of that &quot;beautiful&quot; web design with pretty forms and fonts, bevelled edges, drop shadows, drop-down menus, hovers? Hell, do I even need a map? Heck no, I need three text entry fields and output a bulleted list, maybe table of results. Just give me the raw data and do as little presentation as possible, thanks.I really think I want an internet console, not an animated magazine.

This is really good; I&#x27;ll actually use it!

This is a fascinating tool, I estimated that the corpus of the factual web was between 1 and 10 TB when I last played around with BigQuery using domain names which had low amounts of click bait. Seeing these search results I suspect my estimate was off by a couple orders of magnitude.Although a search for &quot;Fractional Reserve Banking&quot; shows that some further ranking improvements can be made to exclude unrelated results, and potentially penalize old conspiracy sites.<a href="https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=fractional+reserve+banking" rel="nofollow">https:&#x2F;&#x2F;search.marginalia.nu&#x2F;search?query=fractional+reserve...</a>

I love it. Even though it didn&#x27;t give me the results I was looking for. I searched &quot;new york fishing license&quot;, and it didn&#x27;t give me any links to the actual new york fishing license websites. But it did give me a ton of really cute little websites related to lakes and fishing in New York. This one has amazing information about fishing all over Western New York: <a href="http:&#x2F;&#x2F;www.huntfishnyoutdoors.com&#x2F;fishing.php" rel="nofollow">http:&#x2F;&#x2F;www.huntfishnyoutdoors.com&#x2F;fishing.php</a>

The spelling suggestions are presented whenever there isn&#x27;t any results, but sometimes they can be pretty misleading.What happens is that C, as a word, isn&#x27;t indexed because it&#x27;s deemed too short, and the bigram &quot;c strtok&quot; can&#x27;t be found anywhere.Try &#x27;strtok&#x27; instead.

I searched &quot;c strtok&quot; and got one result saying &#x27;&quot;strtok&quot; could be spelled &quot;stroke&quot;, &quot;stork&quot;, &quot;sarto&quot;, &quot;strop&quot;&#x27;.Cool concept though!

I am the first to admit it&#x27;s a pretty dated reference.

&gt; New: You can now look up dictionary definitions for words. If you for example don&#x27;t know what the definition of is is, you can inquire thus: define:is.Oh man, I love subtle jabs and tongue in cheek writing like this. Very Robin Williams-esque.

A search engine that favors text-heavy sites and punishes modern web design