Ashley Williams kicked off her colorful “paint by number” keynote at Node.js Interactive by explaining that npm is actually a for-profit company. Npm makes money by selling its enterprise services and, apart from the amounts required to run the everyday operations of a regular company, its revenue is invested in running the npm registry.
Williams, as the Developer Community and Content Manager, described her job as the person in charge of “explaining how npm works.” Because her audience was probably already familiar with npm as a tool, Williams focused on how it works as a service and some of the staggering figures associated with the registry.
For example, in the 28 days prior to the talk, users had installed 18 billion (“billion” with a “b”) packages from the registry, although this translated to “only” about 6 billion downloads. The downloads are substantially lower than the installs because approximately 66 percent of the installs are now being served from the cache.
The figures regarding downloads are not the only ones that have seen exponential growth. The number of packages is also growing at an accelerated rate. At the beginning of 2015, the registry contained about 12,500 packages. But at the time of Williams’ talk, the number was already up to nearly 400,000. In the week before, 4,685 packages were published in the registry.
Interestingly, the npm repository is also used as a first step to getting into programming for Node. About 160 people publish their first package every week in the registry and, at the current rate of growth, Williams predicts that will increase to an average of 200 people a week through 2017.
Currently, 102,460 active unique publishers are already working within the system, and there are 314,582 registered users. Williams remarks on how amazing this figure is considering the sole advantage of registering on the site is the ability to publish in the registry.
11 lines of code that broke the Internet
Williams also addressed the elephant in the room by tackling the topic of unpublishes. In March 2016, a disgruntled developer unpublished all his modules from the registry. Among them was a seemingly harmless chunk of code 11 lines long — left-pad. left-pad padded out the left hand-side of strings with zeroes or spaces and did nothing else. However, a huge amount of other modules relied on left-pad and broke when the module was removed from the registry, causing no small amount of chaos.
Williams admitted that the left-pad debacle happened because of naive policies at npm. Since, the npm team have devised new policies, the main one being that you are only allowed to unpublish a package within 24 hours of publishing it.
They also hosted a forum on GitHub to get feedback from the community and discovered that most people unpublishing packages were doing so because they didn’t want that package listed on their user page anymore. This led to the new dissociate and deprecate policy. The new policy avoids packages from being erased from the registry, but developers can re-assign the package to npm user. This dissociates the package from the original developer and deprecates it, marking it as unmaintained.
Although Williams admitted that having dissociated and deprecated packages hanging around in the registry is not ideal, it does guarantee there won’t be another random unpublish that will break other people’s setups.
Reliability
So another left-pad won’t happen, but, what would happen if all the registry went down? Williams said this highly unlikely. ping.npmjs.com shows in real time stats the availability of public services npm runs. The site consistently shows that the registry’s services offer a 99.999 uptime.
Williams also pointed out that the registry is also very fast. That’s because the vast majority of the data they need to serve is now served statically. Data is only updated when the registry receives something from the “changes fee.” According to Williams’ benchmarks, downloading from the registry, as opposed to downloading directly from a module’s Git repository, is 75 percent faster.
The registry is also huge. At over 350,000 packages, the npm registry contains more than double the next most populated package registry (which is the Apache Maven repository). In fact, it is currently the largest package registry in the world.
The downside is that 80 percent of npm users are doing front-end development and 20 percent are using npm ONLY for front-end code. Npm was designed for people writing modules in Node and not for developers writing applications or client-side JavaScript. This means that npm’s set of tools are sometimes inadequate for what users want to do.
Fortunately, the community has started writing their own tools to compensate. Williams gave the example of Greenkeeper.io, a service that keeps dependencies updated in front-end applications. Npms, another external service, offers an advanced search of the registry, including metrics. Yarn is especially designed for people who require speedy package installs. It also prevents malicious code from being executed in applications by checksumming the integrity of all installed packages.
Williams pointed out that npm actively supports developers building cool stuff on top of the core services, and she encouraged her audience to check out the registry API documentation and resources like the Replicate service. The latter allows you to see in real time the changes happening within the registry.
Finally, Williams recommends that everybody regularly update their npm package with
npm i npm@latest -g
because the npm provided with the standard Node.js installation tends to be several versions old.
Watch the complete video below:
If you are interested in speaking or attending Node.js Interactive North America 2017 – happening in Vancouver, Canada next fall, please subscribe to the Node.js community newsletter to keep abreast with dates and time.
