Kudos to the Yarn contributors for launching with a public RFC governance model right from the start: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;rfcs" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;rfcs</a>Practical, usable, deterministic lockfiles are wonderful and the lack of them has been the single biggest pain point in the npm ecosystem.

A curl -I shows CloudFlare, at least. npm&#x27;s does not. So it&#x27;s at _least_ doing some sort of cache...

Huh, interesting! I thought this was going to just be a client. I wonder if it resolved to npm&#x27;s servers or something.

Makes total sense, thanks for elaborating! I don&#x27;t do very much node dev, but when I do, it&#x27;ll be yarn from now on.

Oh hi Steve! <a href="https:&#x2F;&#x2F;registry.yarnpkg.com" rel="nofollow">https:&#x2F;&#x2F;registry.yarnpkg.com</a> is just a proxy to the normal npm registry provided by Cloudflare so we can add additional caching and work on network performance (lots of stuff we could layer on top). It should hopefully help those in China as well which would be awesome. But everything you do still lives on the normal npm registry.

At least speaking for myself, the npm package ecosystem is critical and I wouldn&#x27;t want to do anything to undermine the integrity of the npm registry.

hi! they are running a mirror of the npm registry which contains all the package data but none of the permissions&#x2F;user mgmt. this means that they are still dependent on npm infrastructure.npm, Inc strongly encourages people to build and use mirrors. for example, cloudflare (where the mirror lives, and where seb used to work) has been running <a href="https:&#x2F;&#x2F;npmjs.cf&#x2F;" rel="nofollow">https:&#x2F;&#x2F;npmjs.cf&#x2F;</a> for a long time.

ied didn&#x27;t work on windows because of the linking strategy. many people who need to use npm use windows.

It&#x27;s not the first one addressing these problems.Nix and ied (which borrowed from Nix) got these problems pretty much solved.I don&#x27;t understand what spoke against these approaches?I mean okay, nix has its own language, which is probably a turnoff for JS devs, but ied?

It looks like this addresses the biggest issues people have with npm&#x27;s CLI, and it&#x27;s coming from such huge names: Facebook, Google, and Tilde. Reproducible builds are a _huge_ issue, and this gives you that. Looks great!One interesting little tidbit I found from diving into the source:<pre><code> https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;blob&#x2F;master&#x2F;src&#x2F;constants.js#L15

 https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;blob&#x2F;master&#x2F;src&#x2F;registries&#x2F;yarn-registry.js
</code></pre>
It&#x27;s not mentioned in the post, but looks like they&#x27;re running their own registry as well...Oh! And open governance: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;rfcs" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;rfcs</a> !

See also:· <a href="http:&#x2F;&#x2F;yehudakatz.com&#x2F;2016&#x2F;10&#x2F;11&#x2F;im-excited-to-work-on-yarn-the-new-js-package-manager-2&#x2F;" rel="nofollow">http:&#x2F;&#x2F;yehudakatz.com&#x2F;2016&#x2F;10&#x2F;11&#x2F;im-excited-to-work-on-yarn-...</a>· <a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;</a>

Just a heads up: someone tried to comment out the Windows choco instructions with HTML comments but the angle brackets were escaped so the instructions are still there (with the escaped brackets).

Lots of ways you can install Yarn over here: <a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;install" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;install</a>

The yarn team has corrected the blog post[+].[+] <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;599#issuecomment-252948027" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;599#issuecomment-2529...</a>

Could be worth finding the author of &#x27;yarn&#x27; on npm and asking them for the name - it&#x27;s been unmaintained for 4 years:<a href="https:&#x2F;&#x2F;github.com&#x2F;samholmes&#x2F;yarn" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;samholmes&#x2F;yarn</a>

The recommended installation method is actually to use the package&#x2F;installer for your environment. Currently there&#x27;s a Windows installer, Debian package, and RPM package, with Homebrew coming soon. See <a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;install" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;install</a>

I think the write-up should specify `npm install -g yarnpkg` not `npm install -g yarn`.

I hope this becomes the default way to build npm projects... and then npm install becomes yarn :-)  Exactly reproducible builds is a godsend to the js community.

927 was also my first thought. But what made me to reconsider was not more than one big name behind it (but it helped), but the fact that they rely on npm backend and did not reinvent everything from scratch.Basically, yarn is npm client done right reusing the same npm package repo.

That deprives me of the fun of downvoting people mindlessly misusing xkcd.com&#x2F;927.

npm on it&#x27;s own is that bad - this is nothing less than water in the desert.

&gt; Yarn, a collaboration with Exponent, Google, and Tilde.They should mention this at the very beginning. Multiple big players investing in this package manager means that we should maybe inspect a little bit more before chanting xkcd.com&#x2F;927.

Hmm.. is it still available as a configuration option somewhere? I couldn&#x27;t find it on the site.I&#x27;m a huge fan of bundler - it&#x27;s dependency heaven.
I&#x27;m also a big fan of the rubygems repo. It does not allow changes in released versions.Even without symlinks it&#x27;s a much needed improvement in the javascript ecosystem

If you do, would love see experiments on that. I do wish NPM would start adding badges or some other meta data to packages that signify that a module has constraints like:
 - No native code
 - No module state (No multiple singletons, and easily load variants of a module)
 - Pure installation, aka no build steps and is fully cacheable via hashing&#x2F;immutable module patterns
 - Is all pure functional
 - 100% code coverage
 - Etc.

This is something I really want to explore more through less-compatible modes. It was the original way yarn worked, but it wasn&#x27;t compatible enough to be the default mode: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;57" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;57</a>

You can make it work with `node --preserve-symlinks`

Yeah, that would be much better. But unfortunately the node_modules structure (which yarn is attempting to be fully compatible with) makes that impossible.The reason is that each package only finds its dependencies relative to its own location. So your second level dependencies cannot vary from project to project unless you do copying.(Example: AppA and AppB depend on LibX. LibX depends on LibY. Through their deterministic lockfiles, AppA and AppB disagree on which version of LibY to use. There is no way to symlink things together to satisfy that case without copying LibX or altering node&#x27;s package resolution algorithm.)

&quot;Linking: Finally, Yarn links everything together by copying all the files needed from the global cache into the local node_modules directory.&quot;Stop copying stuff. Just make it global and link.Next step is to make packages immutable and signed.I&#x27;m happy with this step and the fact that facebook will be able to push this.

There has been no problems with folder nesting in `npm` in general since the version@3 came out over a year ago.

Have you guys approached the ridiculous folder nesting situation? E.g. breaking out of the current&#x2F;broken node_modules structure?

You forgot the link. :)<a href="http:&#x2F;&#x2F;yehudakatz.com&#x2F;2016&#x2F;10&#x2F;11&#x2F;im-excited-to-work-on-yarn-the-new-js-package-manager-2&#x2F;" rel="nofollow">http:&#x2F;&#x2F;yehudakatz.com&#x2F;2016&#x2F;10&#x2F;11&#x2F;im-excited-to-work-on-yarn-...</a>

Interested to see the emphasis on security, it&#x27;s definitely something that could use more focus in this area. On that note, are there any plans to implement TUF (<a href="https:&#x2F;&#x2F;theupdateframework.github.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;theupdateframework.github.io&#x2F;</a>) or similar into Yarn?

<pre><code> &gt; unless you&#x27;ve actually shipped a product using this tool 
</code></pre>
Given that wycats _worked on_ this tool, I would assume it&#x27;s his experience helping build it that&#x27;s the basis of the claim.(I don&#x27;t work at Tilde so I can&#x27;t tell you if it&#x27;s been shipping with yarn, but it would shock me if it wasn&#x27;t.)

As for performance, Yarn is about 3-5x faster installing all of the dependencies and devDependencies of react-native. The benchmarks are here: <a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;compare" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;compare</a>. It&#x27;s much faster in most scenarios, especially the ones that used to take minutes.

You say &quot;decent performance&quot; and &quot;predictability&quot;. What is the basis of this claim? I&#x27;ve heard these all before, but unless you&#x27;ve actually shipped a product using this tool I don&#x27;t know how you can back this up.

When do you expect ember to transition to yarn?Are there any branches of ember-cli implementing yarn?

I wrote a post explaining why I&#x27;m psyched to be working on it:TLDR:- open, community governance that will support long-term evolution- the technical details get a lot right out of the gate (decent performance, predictability, and security)

Doesn&#x27;t npm effectively replicate bundler&#x27;s functionality (poorly) via shrinkwrap? Why not fix that feature rather than replacing it with a separate utility?

If you look at ruby there is &quot;gem&quot; that you could equal to &quot;npm&quot;. On top of that there is &quot;bundler&quot; that is you could equal to &quot;yarn&quot;.It is the same pattern.

a few points!fwiw, only 2 of those humans work on the CLI, (3 if you count their manager, who spends a lot of time managing). other people work in sales, marketing, engineering on the website, engineering the registry, community + docs, etc.wombat is not a tool for managing our other commercial services! it&#x27;s a way to manage our webhooks product, one of many products we have, though very certainly the smallest.yarn depends on the npm registry, which depends on npm, Inc making money to keep it running.

It&#x27;s not old at all, it&#x27;s first release was 2010.

Because the npm client is a hot mess?The official npm cli is quite old and evolved along with all the different coding styles and architecture choices of the node and JS ecosystems.It&#x27;s also full of dependencies on originally purpose-built modules that suffer from the same problems. This compounds various issues, resulting in long-standing bugs like `npm publish` sometimes not actually including all files in the tar bundle it generates[0].NPM 3 was a partial rewrite but it&#x27;s still built on the same mountain of code as NPM 2 and 1. npm Inc developers have in the past complained about not being able to address issues for a lack of resources despite having a ton of developers on the team[1].The official npm client also doubles as an account management CLI to the npm registry. Which apparently npm Inc now considers a design flaw as they recently started creating a standalone tool for managing other aspects of their commercial services[2].And finally it&#x27;s important to understand that having a new package manager that is not tied into npm Inc means it&#x27;s easier to replace the npm registry as the official node module registry in the future. Which would allow the community to get rid of the privileged position npm Inc currently holds in the node distribution (which is otherwise governed by the Node Foundation).Rewriting the client from scratch actually turned out to be the sanest approach in this situation, I think.[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;npm&#x2F;npm&#x2F;issues&#x2F;5082" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;npm&#x2F;npm&#x2F;issues&#x2F;5082</a>[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;npm&#x2F;humans" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;npm&#x2F;humans</a>[2]: <a href="https:&#x2F;&#x2F;npmjs.org&#x2F;package&#x2F;wombat" rel="nofollow">https:&#x2F;&#x2F;npmjs.org&#x2F;package&#x2F;wombat</a>

Go take a look at npm&#x27;s code and then tell me you&#x27;d want to contribute to that.

This looks awesome. But I have to wonder why create a whole new project rather than fork or upstream these changes to NPM? It doesn&#x27;t seem like it&#x27;s doing anything fundamentally different or outside of NPM&#x27;s scope of responsibility.

Yarn isn&#x27;t a replacement for npm itself. It&#x27;s a client that can read&#x2F;write to npm, and other registries such as Bower.

Would you mind going a bit into detail? I work with npm on a daily basis and I am pretty happy so far.

I find it strange that the time isn&#x27;t invested in already existing projects.But at least it&#x27;s a move away from NPM. I think the most problems I had with JavaScript develompent in the last 2 years came from NPM.

Cargo definitely allows two dependencies to rely on different versions of a transitive dependency. If those deps expose a type from that dependency in their API, you can very occasionally get weird type errors because the &quot;same&quot; type comes from two different libraries. But otherwise it Just Works(tm).

Cargo does allow multiple versions of transitive dependencies. It tries to unify them as much as possible, but if it can&#x27;t, it will bring in two different versions.What it _won&#x27;t_ let you do is pass a v1.0 type to something that requires a v2.0 type. This will result in a compilation error.There&#x27;s been some discussion around increasing Cargo&#x27;s capacity in this regard (being able to say that a dependency is purely internal vs not), we&#x27;ll see.

I feel like this is especially problematic when using NPM for your frontend. Now you have to run all your code through deduplication and slow down your build times or end up with huge assets. I wonder if it&#x27;s really worth the trouble.

This is especially true when you&#x27;re going to be serving your code over the web. It&#x27;s very easy when using npm to accidentally end up shipping a lot of duplicate code.That alone has me super excited about Yarn, before even getting into the performance, security, and &#x27;no invisible dependency&#x27; wins.

As a practical matter, the npm ecosystem today relies on duplication, and no new client that made the &quot;highlander rule&quot; (there can be only one) mandatory could succeed.Yarn does offer a `--flat` option that enforces the highlander rule, and I&#x27;m hopeful that the existence of this option will nudge the ecosystem towards an appreciation for fewer semver-major bumps and more ecosystem-wide effort to enable whole apps to work with `--flat` mode.Plz send halp!

Yarn supports `yarn [install&#x2F;add] --flat` for resolving dependencies to a single version

With tiny modular utilities this is very much necessity - and not a bad idea if the different versions are being used in different contexts.For instance, when using gemified javascript libraries with bundler it is painful to upgrade to a different version of a javascript library for the user facing site while letting the backend admin interface continue using an older one for compatibility with other dependencies.

&gt; Multiple versions [...] almost always a bad ideaIf so, different major versions of the same dep should be considered different libraries, for the sake of flattening. Consider lodash for example.

You&#x27;ve got to take the ecosystem into account. There are a lot of very depended-upon modules that are 1 KB of code and bump their major version as a morning habit. Forcing people to reconcile all 8 ways they indirectly depend on the module would drive them nuts, but including 4 copies of the module doesn&#x27;t hurt much.

Multiple versions may sound like it&#x27;s useful, but it&#x27;s almost always a bad idea. Cargo doesn&#x27;t allow it either.The problem isn&#x27;t really fundamental. Bundler makes almost all the right choices already. Its major disadvantage is that it only works for Ruby.

&gt; Yarn pulls packages from registry.yarnpkg.com, which allows them to run experiments with the Yarn client. This is a proxy that pulls packages from the official npm registry, much like npmjs.cf.[0]Time will tell whether they only want to be proxying NPM or will allow direct pushing to their own registry. If they do, JS ecosystem might see another big shift.[0] <a href="http:&#x2F;&#x2F;blog.npmjs.org&#x2F;post&#x2F;151660845210&#x2F;hello-yarn" rel="nofollow">http:&#x2F;&#x2F;blog.npmjs.org&#x2F;post&#x2F;151660845210&#x2F;hello-yarn</a>

&gt; This also highlights a shrewd move on the part of npm: a clear decoupling between the npm registry and client, with a well-defined protocol between the two. The strength of Node is in the staggering size of its ecosystem; how those bits end up on disk is an implementation detail. This smart separation allows for this kind of experimentation on the client side, without causing the ecosystem fragmentation that happens when a new package manager requires a new registry.And a shrewd move by FB: to not announce their new registry on the same day.

Yes, that&#x27;s why I said it was a Ruby limitation, not a Bundler limitation.

&gt; I loved Bundler&#x27;s deterministic builds but chafed against the Ruby limitation of only having a single version of a dependency at onceThis is due to the way that Ruby includes code, where it&#x27;s available globally, vs Node where code is scoped to the including module. I&#x27;m not sure how Ruby could support multiple versions with changes to the language and&#x2F;or RubyGems

funny (and great!) how cargo became a gold standard for package management without any advertising. well done, rustaceans.

Why do you think that everybody is inspired from javascript tech? It is probably the other way around. For example there were very good build tools long before npm which are still used today and unmeasurably better (like Maven or Gradle).

This is a huge leap forward for the JavaScript community—probably more than many people will realize right away.I loved Bundler&#x27;s deterministic builds but chafed against the Ruby limitation of only having a single version of a dependency at once. npm solved this problem elegantly, but still struggles with non-determinism. I had resigned myself to thinking that maybe these were just fundamental tradeoffs in package management that could never be resolved.Then I had the opportunity to use Cargo, the package manager for Rust. It synthesized what, in my mind, are the best features of npm and Bundler into a single package manager, with terrific speed to boot.Yarn looks like it will be Cargo for JavaScript. After Cargo improved on many of the great ideas of npm, those ideas are returning to help improve the JavaScript ecosystem. Cross-pollination at its best.This also highlights a shrewd move on the part of npm: a clear decoupling between the npm registry and client, with a well-defined protocol between the two. The strength of Node is in the staggering size of its ecosystem; how those bits end up on disk is an implementation detail. This smart separation allows for this kind of experimentation on the client side, without causing the ecosystem fragmentation that happens when a new package manager requires a new registry.I&#x27;m also happy to see that a significant amount of work has already gone into the governance model. Despite the largest contributors being Facebook employees, it looks like they&#x27;ve really outdone themselves making sure this is a community-run project. A BSD license, no PATENTS file, and an Ember&#x2F;Rust RFC process[1]; this has all the hallmarks of a community open source project. That&#x27;s critical for open infrastructure projects like this, and they&#x27;ve nailed it.[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;rfcs" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;rfcs</a>I&#x27;m very much looking forward to using Yarn in my own projects because it looks like it solves a lot of real problems I encounter every day. Thanks for all the hard work!

Similar, but even more similar to Rust&#x27;s Cargo <a href="https:&#x2F;&#x2F;crates.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;crates.io&#x2F;</a>

Am I correct in understanding Yarn works similar to Ruby&#x27;s bundler?

actualyl at this point, unless there is a legal request or a serious security issue, we don&#x27;t allow any unpublishes. we encourage transferring the package to the npm user and deprecating it. yay immutable registry!

If by &quot;social&quot; you mean bad decisions on settling a naming dispute, yes it was a social issue. That issue hasn&#x27;t really been addressed publicly (the official policy still doesn&#x27;t specify how disputes are decided by npm when no amicable agreement can be reached -- other than what essentially boils down to &quot;we&#x27;ll do what we feel is right&quot;).The actual disruption however was solved by npm Inc disabling unpublishing (except for some very specific circumstances) and by creating a dummy account unpublished modules are reassigned to to avoid abuse (whereas previously anyone could simply claim unpublished modules and publish new versions).

leftpad was a social issue. It&#x27;s been solved by policy on the registry side stopping packages depended on by many others being unpublished.

I think it&#x27;s more secure because you can deliver to it (&quot;somehow&quot;) a global cache of packages.For e.g. Facebook, I bet they have some online filesystem like NFS or similar, that hosts the global yarn cache for their continuous integration system. This way, their CI system doesn&#x27;t call out to the cloud for every build.

leftpad wasn&#x27;t a problem, it was a warning that blindly updating your dependencies without acceptance testing is a problem.

Lock file contains hashes of packages, so it works like many &quot;trust on first use&quot; systems (e.g. SSH): if you installed leftpad@0.2.3, you can be sure that futher installations of this same package will be exactly the same — if malicious npm server gives you something else, this should be detected and rejected.Looks like they use SHA-1 hashes, though. Please, let&#x27;s get rid of SHA-1 already.

so, where are the packages stored? how is this more secure than npm? how does this solve the leftpad problem?

&gt;We decided to zip the entire node_modules folder and upload it to an internal CDN so that both engineers and our continuous integration systems could download and extract the files consistently.why not mirror npm modules there then?!

Recently I got annoyed how hard it is to use shrinkwrap in npm and started working on a npm wrapper that would make npm as easy to work with as Ruby Bundler by copying its workflow as closely as possible (<a href="https:&#x2F;&#x2F;github.com&#x2F;szimek&#x2F;bundlerjs" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;szimek&#x2F;bundlerjs</a>). Thankfully, I don&#x27;t have to develop it anymore ;) Big thanks to all Yarn developers!

Just experienced the same. Loved it!  npm was giving vague errors about dependency&#x27;s dependency. Now it&#x27;s so much clearer that which package needs to be updated. Also the speed  of installation has reduced a lot !

i seriously want npm to stop warning me of outdated deps 3 levels down my hierarchy. so what if some shitty old unmaintained lib we started relying on a few years back use lodash v3.x? i&#x27;m not going to worry about it as long as it works.

wow already one of the features i&#x27;m loving in yarn is that it tells you which package is firing warnings about package incompatability.warning electron-prebuilt-compile &gt; electron-compilers &gt; jade@1.11.0: Jade has been renamed to pug, please install the latest version of pug instead of jadein npm, that would have just said the part after &quot;jade@1.11.0&quot; which was really vague and didn&#x27;t really make you want to &quot;fix&quot; it because which npm module do you have to go into? who knows because npm (the package manager) didn&#x27;t tell you.

90% of those files are probably Babel-related.

As I understand it, the related problem is the poor support in build tools for dead code elimination which encourages people to publish libraries that contain a single function like left-pad.

Yeah. Inspecting the node_modules folder for pretty much any front-end project is depressing. Still, I prefer npm to no package manager at all.One problem is that packages often bundle everything, rather than including an npmignore. I&#x27;m guilty of this myself.Do you think that the problem of over-dependence is reversible, and if so, how?

That&#x27;s kind of a red herring when it comes to NodeJS. Webpack and the 50 other tools you need to run it are in a completely controlled environment. Most of the build tooling is working around issues with NodeJS, and NodeJS doesn&#x27;t need to wait for anyone to come up with improved APIs around filesystem interactions and build tooling. This is purely build-time stuff.

Yeah, but we could have a big, standard library polyfill, and it would quickly find its way into everybody cache, just like jQuery.

&gt; That, to me, is what is wrong with npm. The problem stems from node.js not coming with &quot;batteries included&quot; so there is a proliferation of tiny libraries that do the most trivial things.This is in no way a fault of node.js but of the whole JS standard. People these days use npm modules to run in a browser and ship stuff bundled together with webpack (or other bundlers), so even if nodejs had a proper stdlib, you&#x27;d still need to depend on a polyfill so that your stuff works in a browser environment, too.And due to the fact that even if a sane stdlib would ever be standardized, it would take YEARS of time until it reaches significant market share (looking at you, Android, Safari and IE), so you&#x27;d always have to ship a polyfill.

&gt; To make matters worse, each package gets its own node_modules directory. Do a grep for &#x27;async&#x27; in your node_modules. Mine contains ~20 instances of it. Some of those, probably, are the exact same version.Upgrade to npm v3, that should these problems.

To make matters worse, each package gets its own node_modules directory. Do a grep for &#x27;async&#x27; in your node_modules. Mine contains ~20 instances of it. Some of those, probably, are the exact same version.Going further down shit creek, each project I work on has its own node_modules. Which duplicates all of this crap, ad nauseam.Do I really need React and Babel and jQuery and Webpack installed for every goddamn repo? You&#x27;re not a package manager if you&#x27;re not managing packages!! You&#x27;re a glorified wget.

&quot;The React Native package.json currently lists just 68 dependencies, but after running npm install the node_modules directory contains 121,358 files.&quot;That, to me, is what is wrong with npm. The problem stems from node.js not coming with &quot;batteries included&quot; so there is a proliferation of tiny libraries that do the most trivial things.

registry.yarnpkg.com is a proxy, it doesn&#x27;t contain any package data except for what gets cached by Cloudflare. Just a Cloudflare&#x27;d &quot;CNAME registry.yarnpkg.com. registry.npmjs.org.&quot;

that registry is a mirror of npm&#x27;s registry. the mirror contains all package data but does not include any of the permissions or user management, so this tool still depends on npm infra for both that, and as a replication source.

Yarn is currently being powered by a service running at <a href="https:&#x2F;&#x2F;registry.yarnpkg.com" rel="nofollow">https:&#x2F;&#x2F;registry.yarnpkg.com</a> - at least that&#x27;s what&#x27;s referenced in the yarn.lock file that&#x27;s created.I see no documentation nor code for the service powering this, nor is there a way to tell the command line tool to use a registry at a different address.Aren&#x27;t we just replacing the dependency on npm, Inc.&#x27;s registry with a new dependency on Facebook Inc&#x27;s registry with this?Yes. The client has a few advantages, but the registry is still closed and while with npm there&#x27;s an option of running a private registry if you pay them, here, there&#x27;s no option at all.

The &#x27;attempts at scaling npm client&#x27; section is incredibly cringy even to read. Those are real complex problems that every one who works on a massive scale will face at some point in time.

Great !! That sounds like it should help atleast .. hope the heuristics improve to capture most npm patterns

You can run `yarn clean` and it will clear out all the extra files you don&#x27;t need: <a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;cli&#x2F;clean" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;cli&#x2F;clean</a>

In theory, those submodules should be npmignoring those files and subdirectories, right?

I wish some kind of &quot;auto-bundle&quot; feature prevented the creation of multiple hundred MB&#x27;s worth of tests, readmes and docs on disk when something like single &quot;babel.bundle.js&quot; file would do.Is it really important that one knows that a tool dependency uses left-pad ? What could be the drawbacks of such bundling?

Oh Okay. Good one. Thanks for the blog ref #openess

npm&#x27;s been involved since the early days, they wrote up a blog post here: <a href="http:&#x2F;&#x2F;blog.npmjs.org&#x2F;post&#x2F;151660845210&#x2F;hello-yarn" rel="nofollow">http:&#x2F;&#x2F;blog.npmjs.org&#x2F;post&#x2F;151660845210&#x2F;hello-yarn</a>

While Yarn sounds like a nice wrapper with extra stuff. I want to ask if this collaboration couldn&#x27;t happen with the NPM team?

Nice name too: <a href="https:&#x2F;&#x2F;www.getyarn.io&#x2F;yarn-clip&#x2F;6f28eef4-98fe-4ae5-96c0-da3aa65ce319" rel="nofollow">https:&#x2F;&#x2F;www.getyarn.io&#x2F;yarn-clip&#x2F;6f28eef4-98fe-4ae5-96c0-da3...</a>

Rather impressed with this with the few tests I&#x27;ve run. Kudos to the creators and can&#x27;t wait to see this project progress (and to contribute into!) in the near future.

--flat is the cli option, but there&#x27;s also &quot;flat&quot;: true support in package.json. Used at the top level this forces a flat install, but used in a dependency it means that that package requires a flat install and it throws an error if it&#x27;s not.This means that HTML imports and ES6 modules can force flat installs and start moving the front-end package ecosystem in that direction.

This is what I was most curious about, but it looks like --flat is a cli option but not the default. Definitely moving in the right direction though!

Do you mean import &#x27;&#x2F;web&#x2F;X&#x27;? Import URLs have to start with a `&#x2F;`, `.&#x2F;`, or `&#x2F;&#x2F;`.Even so, that requires a build tool. I want to be able to load working sources directly out of my packages directory.

You can use webpack or JSPM to resolve directories at the top level. That way you can just import &quot;web&#x2F;X&quot; even if the web&#x2F; directory isn&#x27;t a sibling to your current file.It&#x27;s not very helpful if you are distributing an NPM package, but for people who aren&#x27;t authoring libraries, its a godsend.

Yarn is particularly great for front-end web apps because of its flat installation mode.ES6 module imports and HTML Imports both require that dependencies are imported by URL. This means that the only reliable way to import another module is by relative URL, like:<pre><code> import * as $ from &#x27;..&#x2F;jquery&#x2F;jquery.js&#x27;;
</code></pre>
This requires that packages are installed flat, as siblings. Yarn is going to enable native JS modules and projects like Polymer to use npm instead of Bower. :)

Note that it doesn&#x27;t work with private modules yet, but it&#x27;s coming soon: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;521" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;521</a>

Elsewhere in this thread, they said that its because it doesn&#x27;t work well with enough of the ecosystem yet.

Overall looks good, but it&#x27;s a shame that flat mode is opt-in instead of the default - this is the sane way of doing package management. Would have been nice to flip the options over so you have to explicitly allow multiple versions of the same package - which you could automatically enable if you&#x27;re converting from npm.

Thank you. Done. The issue is reproducible with an entirely new project.

Sorry about that, there are very likely still some bugs out there we weren&#x27;t able to catch in all our testing which will be ironed out by the community in the next few days. Can you open an issue with details about which package failed to install? <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;new" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;new</a>

npm treats the package.json engines field as advisory, but yarn is strict about it. maybe that&#x27;s you&#x27;re problem

I just tried to install yarn and ran it to against a relatively large React application. What&#x27;s interesting is that it found an incompatible module and could not proceed, whilst npm works fine.The article states that running `yarn` is the same as `npm install`, which doesn&#x27;t look like the case. At that point, I&#x27;m curious what are the differences.

Looks like they are trying to write off yarn as just another alternative install tool. I think this is short cited and hope they are taking this threat more seriously internally.

Does anyone know why npm moves so slowly?

I&#x27;m glad they&#x27;re okay with it but then again what else would they post? npm has some major, major issues with it and it has barely moved in years. This Yarn looks like it solves quite a few issues with npm and it took another company to build it.That&#x27;s insane. It wouldn&#x27;t surprise me if yarn becomes popular and its proxy to npm slowly turns off and boom, everyone would be migrated and npm would be left with little to no users. Yeah that&#x27;s probably unlikely for the immediate future but Yarn is positioned perfectly to siphon off npm&#x27;s users very painlessly.

NPM can evolve; Gradle and SBT also didn&#x27;t turn out to be the end of Maven, which evolved to being more of a repository and dependency management tool with multiple clients.

It is quite a congenial response and no disrespect to npm inc, but yarn will be technically far superior (if it&#x27;s not already), and I can&#x27;t help but interpret this as the beginning of the end for npm....It was only a matter of time though since there are a number of systemic problems with npm and (in my opinion) it is at the center of the dysfunctional world of modern JavaScript tooling. It did move the ecosystem forward significantly for a number of years, but now something better is required. Sorry to be blunt, just my opinion.

Interesting how they point out how Yarn still depends on them since it pulls packages from NPM&#x27;s package repo. Kind of like a reminder of who the dominant player still is.

That&#x27;s pretty well written, though you can see slight hints of bitterness in the tone (though its subtle). Or maybe I&#x27;m a little biased, because I&#x27;ve seen some of the non-public reactions, and they were more than a little bitter...

Here is npm&#x27;s response: <a href="http:&#x2F;&#x2F;blog.npmjs.org&#x2F;post&#x2F;151660845210&#x2F;hello-yarn" rel="nofollow">http:&#x2F;&#x2F;blog.npmjs.org&#x2F;post&#x2F;151660845210&#x2F;hello-yarn</a>

I don&#x27;t know about yarn specifically, but for all the systems it&#x27;s modeled after, the answer is &quot;yes for binaries, no for libraries&quot;.

Docs here [1] say yes.It also seems to imply that this is edited by the `yarn` command so it would be invalidated if you manually edited your `package.json`? Dunno if it&#x27;s destructively invalid though.1: <a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;configuration#toc-use-yarn-lock-to-pin-your-dependencies" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;docs&#x2F;configuration#toc-use-yarn-lock-...</a>

Yes. This makes it part of code reviews and is what keeps everyone who might ever use the project(whether human or CI) using the same libraries.

Should you commit the yarn lockfile to version control?

For what it&#x27;s worth, the name is really cool. I just looked up the etymology of Node and it comes from Latin `nodus` meaning knot. So if a Node program is the knot, then the stuff that makes it up and that it comes from must be the yarn! Language is fun.

&gt; They address pretty throroughly why that does not work for them in the linked article.Not really. They just mention it in passing:From the article&gt;&gt; Shrinkwrap files aren&#x27;t generated by default and will fall out of sync if engineers forget to generate them, so we wrote a tool to verify that the contents of the shrinkwrap file matches what&#x27;s in node_modules. These files are huge JSON blobs with unsorted keys, though, so changes to them would generate massive, difficult-to-review commits. To mitigate this, we needed to add an additional script to sort all the entries.Comparing node_modules to shrinkwrap isn&#x27;t necessary. Building the app will re-generate node_modules and tests should catch whether or not the app works as intended.The sorting issue for shrinkwrap is legit, but the answer to that is sort it! Sure you&#x27;ll get some deep diffs if the dependencies change, but that reflects the real changes to the app.The simpler answer seems to be to add a build test to mandate the existence of a shrinkwrap file, verify it&#x27;s sorted, and (optionally) verify it&#x27;s up to date. The latter doesn&#x27;t even have to be done every build.

They address pretty throroughly why that does not work for them in the linked article.

&gt; Finally, updating a single dependency with npm also updates many unrelated ones based on semantic versioning rules. This makes every change much larger than anticipated, and having to do things like committing node_modules or uploading it to a CDN made the process less than ideal for engineers.You shouldn&#x27;t check in node_modules. You should use a shrinkwrap file to indicating the specific sub-dependency versions you&#x27;re using.

I&#x27;m so excited about this. Having wasted lots of time migrating Shopify to use npm for JS package management, it seems that yarn fixes everything that had been a nuisance with it. Woo!

As Sebastian said we are more than excited to explore what we can improve in the future. If you are interested, please start a discussion on the GitHub issue tracker: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn</a> :)

&gt; We initially used a method similar to ied when we first started experimenting with internal usage at Facebook.Would be really nice if you would e.g. add ied to &quot;Prior art&quot; on your README then. ;)&#x2F; @alexanderGugel (the guy behind ied)

Hey! I&#x27;m Sebastian McKenzie (@kittens on GitHub) and I&#x27;m the lead develop on Yarn at Facebook. We initially used a method similar to ied when we first started experimenting with internal usage at Facebook.We ran into a lot of issues with OS compatibility around symlinks and existing tools not supporting it. Windows lacks support for symlinks on non-Admin accounts, NTFS junctions work but they&#x27;re slightly quirky in their behaviour and don&#x27;t behave exactly the same way.A lot of tooling relies on the existing node_modules structure. ESLint for example will load tools relative to itself which means that they need to be reachable in the tree, you can solve this with hardlinks but there&#x27;s caveats with that too. Existing tools aren&#x27;t very well suited for handling cycles either which in a system like npm are extremely common as a transitive module could depend on one that&#x27;s higher in the graph.We care a lot about existing ecosystem compatibility and because of these limitations we went with the current status quo. We&#x27;ll continue to explore others options though and if there&#x27;s a chance we can do it without any of these caveats then we&#x27;re more than happy to explore it.Let me know if you have any further questions!

My initial tests with this are very positive (I tried this use case here: <a href="https:&#x2F;&#x2F;github.com&#x2F;npm&#x2F;npm&#x2F;issues&#x2F;10999" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;npm&#x2F;npm&#x2F;issues&#x2F;10999</a> ). It handled it quite well and we&#x27;ll definitely be considering this for our package.I&#x27;m curious about the &quot;flattening&quot; approach though -- and why something like ied ( <a href="https:&#x2F;&#x2F;github.com&#x2F;alexanderGugel&#x2F;ied" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;alexanderGugel&#x2F;ied</a> ) wasn&#x27;t done instead. ied only ever installs one copy of a module, but allows each module to think they have their own copy of it through clever symlinkery (essentially making it FEEL like npm 2 while getting the benefits of npm3&#x2F;yarn). I&#x27;m not a fan of (explicit) flattening for a number of reasons:1. Mimicking recursive dependencies while still only having on physical copy on disk has the benefit (or downside I suppose) of isolating and protecting different module&#x27;s dependencies from each other. AKA, if you require(&quot;x&quot;).mutate = true, this won&#x27;t be seen by other modules. Again, depends whether you like this feature or not, but with node&#x27;s treat-symlinks-as-different-files configurable switch, you can actually control it yourself in ied I believe.2. You don&#x27;t run into tricky situations where require(&quot;x&quot;) can accidentally pick something up you didn&#x27;t mean to (since it is now in the parent path of ALL your dependencies). So, if A depends on x, and you install B, it will be able to require(&quot;x&quot;) (maliciously?) and modify it, affecting A&#x27;s behavior. Perhaps this is considered a strange possibility, but it gets particularly weird with peer dependencies, where you expect the user to choose their own top level dependency, but this can now be accidentally &quot;chosen for you&quot; by a subdependency of another package.3. I just don&#x27;t like clicking on my node modules folder and seeing 100 top level directories when conceptually I typed &quot;pkg-manager install one-thing&quot;. This can lead to you yourself making the above mistake, by require(&quot;x&quot;) and having it &quot;just&quot; work, then having it &quot;just break&quot; when you remove the dependency that included x.I&#x27;m willing to be convinced though (on RunKit we had module-fs simulate npm2 install behavior, super simple recursive installation, conceptually similar to what ied does).

Brilliant. I&#x27;ve been trying to tackle this problem for the last year or so with shrinkpack (<a href="https:&#x2F;&#x2F;github.com&#x2F;JamieMason&#x2F;shrinkpack" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;JamieMason&#x2F;shrinkpack</a>) with _some_ degree of success and a nagging feeling that much more could be done.It&#x27;s great to see attention is being spent on what is a really important area, one which has been largely under-appreciated in Node.js until now (aside from for a short period after the left-pad incident).Kudos to Facebook, I look forward to giving it a try.

They are different problem domains, I think confusing them is unlikely

As someone who just started with Spark lately, that was my first thought after being confused for a minute. It&#x27;s not like people at facebook and google don&#x27;t know about Apache projects. I wonder what is their motive to not avoid such name collision? I see these clashes a lot with small projects on github -ironically most of the are javascript- but rarely on a big scale like this.

generate name
compare affinity in concept space
if affinity &gt; .5
 repeatyarn is framework for doing X in web programming, and this a framework for doing Y in web programming. not sure if dependency management and cluster system management are far apart enough.

Clashes with Apache YARN - <a href="https:&#x2F;&#x2F;hadoop.apache.org&#x2F;docs&#x2F;r2.7.2&#x2F;hadoop-yarn&#x2F;hadoop-yarn-site&#x2F;YARN.html" rel="nofollow">https:&#x2F;&#x2F;hadoop.apache.org&#x2F;docs&#x2F;r2.7.2&#x2F;hadoop-yarn&#x2F;hadoop-yar...</a>

This is a very good step in the right direction.It still doesn&#x27;t solve the biggest problem the node.js and javascript ecosystem has IMHO: dependency hell, thousands of trivial packages, just utter chaos.

Just an easy way for javascript devs to add big data keywords into their CV. :p

<a href="https:&#x2F;&#x2F;twitter.com&#x2F;k4rliky&#x2F;status&#x2F;785875228710858753" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;k4rliky&#x2F;status&#x2F;785875228710858753</a> please no

What does this mean for NPM Inc the company?Is a replacement &#x2F; mirror &#x2F; rethink of the NPM infrastructure coming next?If I was NPM Inc I would think about adopting yarn if possible.

You are right, we aren&#x27;t 100% compatible right now. Please track the issues on the Yarn GitHub repo that you linked or better yet: help us fix it. This is a community project and we are excited for the community to jump in and help us out and complete the few missing pieces.I agree local development with many packages can be hard. For Jest we adopted `lerna`See <a href="https:&#x2F;&#x2F;github.com&#x2F;facebook&#x2F;jest" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;facebook&#x2F;jest</a> and <a href="https:&#x2F;&#x2F;github.com&#x2F;lerna&#x2F;lerna" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;lerna&#x2F;lerna</a> which makes cross-package development in a mono-repo a lot of fun. Maybe that solution will work for you?

This. I came here to say this also.
In ruby if you want to patch a repo for yourself you just fork it and make changes and then in bundler point to the git url (often on github). Then run bundle install and you&#x27;re set.With npm I&#x27;ve been frustrated a few times whereby you can fork the repo and point package.json to the git repo but after running npm I usually get a bunch of compilation errors.It would be wonderful to make this as simple as with bundler within the ruby ecosystem, point to forked url and running yarn handles the rest.

I&#x27;m very excited about this. Private package support is a must-have for it to get any traction in my company. We are more than ready to leave shrinkwrap behind.

Yeah, this is like the main reason I love npm: ability to use a Github repo as a package repo.

There&#x27;s a lot to like here - deterministic builds are great. However, yarn doesn&#x27;t currently seem to support installing straight from github or private packages [0], [1].I&#x27;m sure this will be added in the future, but it is currently a dealbreaker for me - I&#x27;m using a single package that&#x27;s just straight hosted in a private git repo, as the package itself isn&#x27;t ready to be published yet. I&#x27;m sure other people are using private packages for more normal reasons (closed source, pre-production, etc).If yarn actually made it simpler to refer to a local package during development, I&#x27;d be on board with that. (I am developing the dependency, but want to also work on the thing that depends on it, so I&#x27;d rather just save locally and refresh npm on the outer project, but that&#x27;s hard to get right - file urls don&#x27;t always update, the easiest workflow seems to be delete the package from node_modules, and npm install, with the package having a git url rather than a version in package.json).0: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;573" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;573</a>1: <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;521" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;issues&#x2F;521</a>

Pretty close to <a href="https:&#x2F;&#x2F;github.com&#x2F;discourse&#x2F;discourse&#x2F;blob&#x2F;master&#x2F;Gemfile.lock" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;discourse&#x2F;discourse&#x2F;blob&#x2F;master&#x2F;Gemfile.l...</a>

yarn.lock[0] is similar to composer.lock[1][0] <a href="https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;blob&#x2F;99dd4504469273332f01ce5ad846c995378cc048&#x2F;yarn.lock" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yarnpkg&#x2F;yarn&#x2F;blob&#x2F;99dd4504469273332f01ce5...</a>
[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;composer&#x2F;composer&#x2F;blob&#x2F;20ee689bb464edfe7e57af6be8f03fb664f5db1d&#x2F;composer.lock" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;composer&#x2F;composer&#x2F;blob&#x2F;20ee689bb464edfe7e...</a>

I don&#x27;t agree with any of your statements, and I think they are inaccurate.jspm aims to be type-agnostic with respect to modules (supports all major types, hence why it is called a universal module loader). jspm certainly doesn&#x27;t advocate rewriting your modules (you can use exports, AMD, ES6, whatever). In fact, when the browsers natively support module loading, systemJS will go away (which is really just a polyfill for this functionality).jspm also has npm compatibility.

I don&#x27;t think it&#x27;s fair to compare jspm and Yarn. jspm is an interesting project, but is tightly coupled with SystemJS. Telling everyone to rewrite their apps to use SystemJS and jspm is a huge lift.Yarn, on the other hand, has focused on npm compatibility from the get-go. The entire point is to be a drop in replacement, with no changes (or maybe very very minimal changes) required to gain a lot of benefit.In this slice of the industry where everyone is told they have to rewrite to the latest thing every year or two, I find it very refreshing to see Yarn&#x27;s focus on backwards compatibility, improving existing infrastructure, and having a clear, painless adoption path.

Its disheartening to see so many duplicate efforts to solve the same problems everyone faces. Instead of supporting an existing open-source project that attempts to solve the problem in pretty much the same way (jspm), a conglomerate once again builds their own from scratch. I do not think the underlying motivations for doing so are questioned enough. Sure there is control, but it is of course anyway-you-slice-it an aggrandizement of their brand. We should be ashamed as a community to support the idea of yet ANOTHER client side dependency management system. How oh how did we ever solve this problem before Facebook came along and made Yarn in late 2016?

JavaScript is the new BASIC and every millennial wants to populate his&#x2F;her Github portfolio for Silicon Valley interviews.

A very limited standard library with no native (well, just approved but not ratified) support for importing&#x2F;libraries. Coupled with inconsistent implementations and you end up need more layers of abstraction.

Browsers are the new OSes. Everything on them runs on javascript. You do the math.

I came to the Javascript world late, starting with React. I think its tooling has gotten out of hand, but there are a couple of reasons it&#x27;s not a _total_ disaster in my mind:* Partly, there are a larger number of Javascript developers that haven&#x27;t been exposed to other ecosystems, and end up reinventing some wheels* More importantly though, I think the pace of innovation in the javascript ecosystem is actually good in a way, because other more mature languages (I&#x27;m not talking about age of the language, just maturity of the tooling and ecosystem) have gone through the same growing pains, it just took a lot longer. It&#x27;s easy to forget, coming from the Python world for instance, the pains of easy_install and distutils vs seuptools, etc etc. I still find Java dependency management a bit of a pain, because I don&#x27;t know it very well.We&#x27;re just watching a sped up version of a language maturing, and it&#x27;s painful as individual developers trying to keep up, but I don&#x27;t think it&#x27;s as negative as HN often makes it.

Your number 3 is the one that really drives it home for me. It&#x27;s the unix philosophy taken to the extreme, and (at least in my experience) it&#x27;s working very well.It does take some getting used to, but when you stop looking for one big tool that can solve 5 semi-related problems, and start looking for 5 little tools that each solve one of the problems, things get much more clear.And yes, I know that this kind of reliance on dependencies can cause issues, but from my experience, those issues don&#x27;t seem to bite that often, and many of them can be resolved with even more tooling.

browser incompatibility is the biggest issue (if not the only issue). this is almost a non-issue if you&#x27;re just writing node related programs. but if you need to support in the browser, now you have edge, chrome, firefox, and safari at minimum. then some have to continue supporting ie10&#x2F;ie11, or even worse, ie7+.don&#x27;t forget about mobile devices, with ios having such a difficult-to-write-anything-new version of mobile safari.i&#x27;m writing a new app for myself, so i don&#x27;t have to worry about any backwards compatibility issues, except it needs to work on my iphone, which uses mobile safari webviews, which means i have to either use shims or restrict myself back to es5, non-html5 features (or work around their html5 limitations).example: you cannot cache audio files to play using &lt;audio&gt; tags in mobile safari. safari just refetches it from your server (also has to recreate the audio object, which causes a lag in the audio being played). technically you _can_ cache if you convert all your audio files to base64 and use the webaudio api. i&#x27;m probably going to have to go this route because i&#x27;d like the app to be offline usable.so rather than spend time on my app, i now have to write out a build step for converting audio to base64. then test that it works fine in ios and desktop browsers (at least chrome, firefox). it all just keeps adding up.

Basically I think it comes down to:1. In the browser we have less control over the environment the code runs in, requiring tooling to achieve a stable, normalised environment which we can improve at our own pace, rather than the rate at which browsers are updated.2. JS has become very widely used very quickly, which means that there have been lots of hastily built things, which have been replaced by better things (and for every &#x27;better thing&#x27; there are many alternatives which fell by the wayside), and a larger pool of people to build them.3. It&#x27;s easier than ever to build new tools (or any kind of app, for that matter), because the npm ecosystem makes it trivial to combine existing smaller pieces of code into new combinations.

Javascript in my opinion is still very young. You can&#x27;t compile it to web assembly (yet) and the fact that there are different browsers that interpret it in so many different ways make for libraries to exist to mitigate this. Also none of these tools are 100% necessary, they just make your life easier and tools always have room for improvement. I&#x27;m sure others can add to this. I don&#x27;t think any of this is a bad thing, it&#x27;s just overwhelming for newcomers that haven&#x27;t been following this whole thing.

Why is that?
In the Java ecosystem there is Spock framework that gives you everything required for testing, you don&#x27;t even need junit+mockito.
In which scenario would be better to use 11 different libraries for doing the same thing as a well written framework?

&gt; It&#x27;s a much newer technology than javaStrangely enough both Java and JavaScript were first released in May 1995. Server side JavaScript was released the same year just after the browser version.Though I get that you are referring to the js eco systems (commonjs, npm) compared to the much more mature Java ecosystem.

I have no idea if you&#x27;re familiar with the java ecosystem, but I am, so I&#x27;ll use that as an example:Node libraries are typically several orders of magnitude smaller and more single purpose than java libraries. Java libraries like Guava and Spring are typically quite large. A practical example is from unit testing: In Java, you might use JUnit+Mockito for your test dependencies. In Node, to get similar functionality, I have the following deps: Mocha, Sinon, Sinon-As-Promised, Chai, Chai-Moment, Chai, Chai-Things, Chai-Shallow-Deep-Equal, Chai-Datetime, Sinon-Express-Mock, Supertest.Many Javascript applications both client and server functionality, and there is different tooling for both. Webpack is currently popular for building stuff that will run in the browser, whereas server side stuff is different.It&#x27;s a much newer technology than java and there is quite a lot of churn around tooling in general as the nature of the way applications are built change.

Is npm the bazooka or C++? As someone who&#x27;s worked with C&#x2F;C++ (and its hilarious explosion of tooling -- half-a-dozen at Google alone), maven, bundler and Cargo, I think it&#x27;s just not accurate to say that the npm ecosystem has a particularly high level of complexity.

Compared to C&#x2F;C++ tooling and package managers, npm is like taking a bazooka to a knife fight.

You say tooling, but then talk about libraries.Because, from this outsider&#x27;s perspective, it seems like most of the tooling is library management (rather than, say, profiling or linting -- does javascript even have a linter? -- or debugging).

You say tooling, but then talk about libraries. I think both are just as numerous for nearly all popular programming languages. It’s just that JavaScript is currently “hot shit”, so all this stuff is a lot more visible.There are tons of libraries for C, many of which aim to do the same, only faster&#x2F;smaller&#x2F;scalable&#x2F;whatever. There are many compilers, both open and closed source. And there are documentation tools, syntax checkers, unit testing frameworks and whatnot.Of course there are also languages like C# where there’s a de-facto standard for almost everything: IDE, compiler, base library, GUI framework(s).I think choice is great!

41 dependencies isn&#x27;t even that much.Babel, the most popular transpiler, is 335 total dependencies, and it doesn&#x27;t even do anything out-of-the-box. You need to add babel-preset-latest for 385 total dependencies if you want to actually transpile anything.Want a linter? ESLint is 128 total dependencies.

This is a valid question coming from an outsider looking in.There are various packages that are extremely small (see leftpad and associated controversy). They often wind up in a slightly larger package, which itself winds up in a slightly larger package, which recurs until you finally wind up with one actual package of consequence that you&#x27;re targeting. For example, Express (a very common Node.js abstraction) has 26 direct dependencies, yet 41 total dependencies.A lot of this results from early Node.js&#x27; mantra of having small, focused packages. This could potentially be a good thing because instead of having three major dependencies that have their own way of leftpadding they can all rely on one library and thus code is efficiently reused. This can be bad, however, when they each need their own version of the dependency - or worse - the dependency is removed from the registry (see Leftpad and associated controversy).One of the legitimate problems that I&#x27;ve seen is that there are various libraries that are just thin wrappers over functionality that is native to Node.js - but written to be &quot;easier to grok&quot;. Thus, these thin abstractions become practical code duplication out of some mixture of Node&#x27;s developers lack of effective verbosity&#x2F;documentation and application developer laziness. But then they creep higher up the dependency chain because someone at a lower level used it.On one hand it can be quite simple (and rewarding) to write code while minimizing dependencies. On the other hand, abstractions such as Express make it very easy to write code and it feels like you only have &quot;one dependency&quot; until you look under the hood.

What do you find maddening about micro packages? I hate depending on a giant stack of frameworks (slowing installs, builds, adding behavior surface area) when I just need a function or two, so micro packages are a joy for me.

&gt; Are people really making projects with dozens (hundreds? more?) of dependent libraries?Yes.&gt; Are there aspects of the language or runtime that reward multiple layers of configuration management?A significant fraction of the node community likes to split their project into really, really small packages and treat them as if they&#x27;re independent even if they come from the same source repo and tend to get used together. As an example, the popular collection processing library Lodash publishes a package for each method and people actually do depend on individual methods.There are two major rationales for this. The first is that JS is a dynamic language and in dynamic languages, smaller chunks of code are easier to reason about than larger ones. The second is that the JS community as a whole cares more about artifact size than pretty much any other community. Having smaller packages allows you to produce a smaller payload without having to rely on techniques like tree shaking.I find micro packages maddening but people give me their code for free and it mostly works so I don&#x27;t complain about it that much.

Others have touched on the subject but one of the biggest issues is browser compatibility.  Building a responsive website that works on all major browsers, across multiple versions, as well as all the mobile quirks is not an easy task in the slightest ...  and that&#x27;s just to make a site look pretty.  Then we have to start talking about building massive web apps that are front-end heavy which requires even more tooling to scale, maintain properly.

That&#x27;s interesting because I have less JS headaches when using Node. I avoid front end JS these days

A lot of these answers are pointing out the x-browser complexity which is true but that isn&#x27;t what drives JS package mgmt. It&#x27;s mostly Node stuff. The mature common JS libs for browser work are pretty robust when it comes to x-browser stuff now days.For browser work, you can do almost everything you need with some jQuery, handlebars and moment if there are dates. A few other small libs here and there and you&#x27;ve got almost everything covered. Some edge cases may drive you to use some library that is heavily dependent on 32 different sub-libs but it&#x27;s really not that often.Server-side Node.js is the issue, not browser compatibility.

Let&#x27;s consider Microsoft Office. It is a massive software and such as massive software requires a lot of (internal) dependencies and toolings.Now imagine Office 365, the same software with the same functionalities, except running in the browsers. The task is to essentially write Microsoft Office (front-end logic) entirely in JavaScript (or TypeScript), with the additional problems of dealing with browser compatibilities issues, which are just as hard as OS compatibilities issues.Then, imagine all the software going to the &quot;cloud&quot;. JavaScript now has to have the capability of re-writing all these software originally written in C++, C#, Java... all sorts of languages. It is quickly becoming a superset of all ecosystems where everyone is trying to migrate their software to. So it is not surprising that people are inventing more and more tooling to make it more powerful to suit their needs.

&gt;Are people really making projects with dozens (hundreds? more?) of dependent libraries?Yes, because the standard library provides practically nothing useful in itself.

The tooling for building C++ consists of make or something that generates make files. There&#x27;s also no &quot;1 function&quot; libraries.
Adding a dependency is done by (compiling it if needed) and referencing it from the make file. 
It&#x27;s super easy to understand, but involves manual work because C++ doesn&#x27;t have a package manager.Java isn&#x27;t too complicated either. I&#x27;ve used Gradle and Maven but doing things manually like in C++ can also work.Crucially, they don&#x27;t reinvent the wheel every couple of months so a C++ or Java dev can focus on building things instead of relearning how to build things.

Handling dependencies is hard. Handling dependencies for a platform you don&#x27;t control is harder. Handling dependencies for a platform you don&#x27;t control, where every single kilobytes count, is insane.Add those together, and you get the need for real complex tooling.With that said, tooling is nowhere as complex as it is for languages like C++ or Java. People are just used to it on those platforms, and the tooling is more mature&#x2F;understood because it already went through the countless iterations, 15-20 years ago or more.

You should check out the mayhem caused by a single contributor removing his left-pad library.<a href="http:&#x2F;&#x2F;www.theregister.co.uk&#x2F;2016&#x2F;03&#x2F;23&#x2F;npm_left_pad_chaos&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.theregister.co.uk&#x2F;2016&#x2F;03&#x2F;23&#x2F;npm_left_pad_chaos&#x2F;</a>

Because JS is open source and it&#x27;s not controlled by an Oracle &#x2F; Microsoft &#x2F; Apple &#x2F; XXX foundation. Everybody is free to do stuff - and does. In the end there aren&#x27;t actually as many tools as people think, a lot of hobbyists projects who die quite quickly

This may come off as a troll, but it&#x27;s an honest question. I&#x27;m not a javascript guy. It&#x27;s not a language I deal with at all. Why on God&#x27;s green earth does it need as much tooling as it seems to have? Are people really making projects with dozens (hundreds? more?) of dependent libraries? Are there aspects of the language or runtime that reward multiple layers of configuration management? In short, what the hell is up with this ecosystem?

Sure, not saying there aren&#x27;t any benefits but as a casual user they don&#x27;t feel big enough to justify a switch. I may be wrong though (hence why I&#x27;m asking).

You don&#x27;t? Consistent and reliable dependency versioning across all machines, and speed:<a href="https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;compare" rel="nofollow">https:&#x2F;&#x2F;yarnpkg.com&#x2F;en&#x2F;compare</a>

Depends on how casual :P But probably not.(Edit: Note: I mean you don&#x27;t _have_ to. You might like the performance increase, but if you only use it casually, even that might not be worth the trouble.)

So, as a casual user of node... Should I switch? I don&#x27;t see any immediate benefits.

Just installed it and ran it on a largish project ( <a href="https:&#x2F;&#x2F;www.bugreplay.com" rel="nofollow">https:&#x2F;&#x2F;www.bugreplay.com</a> ), the parallel downloads really sped things up and outputted some great diagnostic info about problematic dependencies. BugReplay is a webapp for recording webapp bugs, so having some confidence that our dependency chain isn&#x27;t introducing our own bugs would be awesome :)

As I understand it, it automatically locks in the (exact) version of everything you install, rather than having this as a separate, optional feature (like shrinkwrap does, despite it not working well - but that&#x27;s an implementation problem).The big deal about semver and how it&#x27;s implemented in NPM is that there is a choice of how to deal with it - either you automatically allow compatible updates and risk an occasional maintainer fuckup, or you explicitly pin everything and take on the burden of constantly updating your versions manually and tracking security releases.The latter might work for Facebook and other large organizations, but for many smaller developers, the consequences of an accidental break are much, much less serious than those of constantly having to keep pinned dependencies up to date. End result: people don&#x27;t bother at all, and insecure versions never get upgraded for the majority of the userbase.This, in turn, removes the incentive for package authors to do anything with semver at all, once Yarn&#x27;s approach becomes commonplace. Why bother with semantic versioning if everybody is expected to manually check and pin their dependencies anyway?Facebook also needs to take into account their influence over the ecosystem. The reality is that whatever project Facebook puts out is immediately seen as gospel and &quot;naturally, a well-engineered thing and a good choice to use!&quot; - this is literally a daily argument in #Node.js when somebody is trying to argue why a poorly-fitting tool is somehow appropriate for their usecase. &quot;Well, Facebook uses it!&quot;In reality, the Yarn workflow is designed to work for Facebook and organizations of similar size, and in all likelihood it will break down in other situations. People will not realize this until they&#x27;ve already bought into it, and by then it&#x27;s too late.

Well, this is a problematic case of &quot;throwing out the baby with the bathwater&quot;. Semantic versioning isn&#x27;t the problem here (the lack of deterministic install trees is!), yet it&#x27;s thrown out pretty much entirely, without even so much as considering why it exists in the first place.I wonder whether Facebook realizes just how much damage they are going to be doing to the JS ecosystem with this approach.

It&#x27;s unlikely npm would accept patches moving it in the direction Yarn is taking.  Different priorities.

Why another project?  If npm has been suitable and only falls short in certain cases, why not contribute back to it?

The algorithm sounds very much like how Dart&#x27;s &quot;pub&quot; command works. Anyone know how they differ?

<pre><code> console.log(&quot;u got powned&quot;)</code></pre>

&gt; The javascript &quot;ecosystem&quot; takes the unix philosophy to the extreme in many ways.Bullshit, nix systems rely on stable global dependencies, not 50 versions of the same library installed locally. I don&#x27;t buy the &quot;unix philosophy&quot; excuse. The nodejs community just doesn&#x27;t care about API stability which is why there is that inflated number of modules with 90% never maintained more than a year.

IMO it&#x27;s the &quot;javascript way&quot; to invent new tools instead of trying to improve others (when it makes sense to do so).Let me explain that a little bit... The javascript &quot;ecosystem&quot; takes the unix philosophy to the extreme in many ways. And one of the points of the unix philosophy is to try to avoid &quot;bloating&quot; tools with tons of options, and instead trying to create new tools where appropriate.Many of those are perfect examples of where &quot;improving&quot; a current tool with the changes they wanted would end up with a more complicated and difficult to use tool.- NPM makes tradeoffs that the majority of developers in the JS world want right now. Like it or not, deterministic installs isn&#x27;t something that most people want or need (at least, they don&#x27;t know that they might want or need it). Making a new tool that anyone can use when that is needed is a great solution, and it avoids bloating NPM with a hundred little flags and tons of code to cover edge cases. It&#x27;s the perfect example because anyone can switch to using yarn when it&#x27;s needed without any extra work.- Nuclide is actually just a layer on top of Atom. They wanted a bunch of things to all work one way, so rather than try to get Atom to change to include what they wanted, or require developers install a ton of extensions, they packaged them up on top of Atom as one system.- Jest had different goals at the start. Testing react components was ugly and difficult, and Jest wanted to solve that first (IIRC, i&#x27;m not very familiar with jest unfortunately). Again, it&#x27;s something that can be used if needed, and can even be combined with other testing tools if you want (we almost did that at my work, used Jest to test the front-end components, and our current mocha+friends setup to test the &quot;business logic&quot;).

&gt; Do you think Flow was built in 3 months as a reaction to Typescript?They probably knew that TS was in progress when they started, but I don&#x27;t have a window into that. I did ask some Flow devs a few weeks ago if they are interested in merging with TS, and they answered with a categorical no.

TypeScript was released October 1, 2012. Flow was released November 18, 2014Did Flow take 2 years to get implemented? I didn&#x27;t think the initial release looked like a project that had 2 years of dev time on it.

&gt; Flow, introduced 3 months after TypeScriptDo you think Flow was built in 3 months as a reaction to Typescript? Not that it was a big enough problem that two different groups of people independently decided to try to solve it?
EDIT: disregard this, Flow was introduced much longer than 3 months after TypeScript. The blog post introducing Flow mentions TypeScript, and is probably worth reading for background [1]&gt; Nuclide instead of Atom&#x2F;Sublime&#x2F;VSCodeNuclide is an Atom plugin.&gt; Jest instead of Jasmine&#x2F;MochaJest provides a superset of Jasmine functionality, and uses Jasmine as its test runner.[1] <a href="https:&#x2F;&#x2F;code.facebook.com&#x2F;posts&#x2F;1505962329687926&#x2F;flow-a-new-static-type-checker-for-javascript&#x2F;" rel="nofollow">https:&#x2F;&#x2F;code.facebook.com&#x2F;posts&#x2F;1505962329687926&#x2F;flow-a-new-...</a>

+1 for contribution to npm. Especially given npm isn&#x27;t completely deterministic there is no reason for not pushing for these changes into npm itself.Given npms years of crap I&#x27;m ready to give it a shot anyway. This might be one of FB&#x27;s projects that actually gets some traction, especially given its developed in collaboration with other big names.

When you&#x27;re game changer, it&#x27;s hard to make current projects maintainers nod their head for you!!! I think io.js was a good example.

I for one am thankful for Jest and Yarn. I recently switched over to Jest and I love most everything about it and the built in watch with caching reduces friction immensely (now if we can just allow custom preprocessors to return promises that would make life easier for integration of things like rollup). And yesterday I flipped over to using yarn and it definitely solves most of my pain points with npm.So yeah, I&#x27;m definitely on the &quot;keep reinventing&quot; side of this debate.

Hey, I work on Yarn and Jest. Let me respond to why we built Jest: It solves a ton of problems no other test runner solves and it works well at Facebook&#x27;s scale. We open sourced it more than two years ago but only recently started to promote the project in open source more actively because it is working so well at FB.If you are curious to find out more about why we built it, please read our blog posts, beginning with one about testing performance and our unique constraints at FB: <a href="http:&#x2F;&#x2F;facebook.github.io&#x2F;jest&#x2F;blog&#x2F;2016&#x2F;03&#x2F;11&#x2F;javascript-unit-testing-performance.html" rel="nofollow">http:&#x2F;&#x2F;facebook.github.io&#x2F;jest&#x2F;blog&#x2F;2016&#x2F;03&#x2F;11&#x2F;javascript-un...</a>Finally, test runners tend to get stale. Any large organization will run into trouble with any kind of tooling as companies scale up engineers or codebases. Testing is no different and we aimed to solve those problems for Facebook; getting to open source our solutions is an added bonus and we are excited to work with the JS community to build the best tooling we can possibly build.

I use JS+Node+NPM for my day job and many side projects.Initial thoughts:- Why didn&#x27;t Facebook contribute the updates to NPM directly?- They are coupling a package manager and registry proxy; the latter has many existing implementations already- The differenced between Yarn and NPM+Shrinkwrap do not seem substantive; NPM made the design decision to use a sometimes non-deterministic install algo in NPM3 to speed up install times - when network is involved, there is a serious trade off between idempotency and speedIn general, FB seems to love building their own versions of existing tools:- Flow, introduced 3 months after TypeScript- Nuclide instead of Atom&#x2F;Sublime&#x2F;VSCode- Jest instead of Jasmine&#x2F;Mocha- DraftJS instead of (insert of of the existing 100 text editors here)- ...I get that these are useful internally at FB, but I don&#x27;t think they help the community much. It would be better to work with existing tools and contribute back to them than to reinvent the wheel every time something doesn&#x27;t work perfectly for their use case.I get that FB uses these as recruiting tools, it&#x27;s useful for them to have rights for these projects, and it gives their engineers something to do and be excited about, but I do not want a whole dev tool ecosystem controlled by big FB.Also, I find IED&#x27;s approach to speeding up builds far more novel and interesting - <a href="https:&#x2F;&#x2F;github.com&#x2F;alexanderGugel&#x2F;ied" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;alexanderGugel&#x2F;ied</a>

Yarn – A new package manager for JavaScript