all-the-packages
All the npm registry metadata as an offline event stream.
Why?
This package in an attempt to make the npm registry's metadata more accessible to a
broader group of developers by encapsulating the entire registry in a single
node module with a simple streaming interface.
There is a ton of useful package metadata in the npm registry, but if
you're not an expert with CouchDB it can be pretty difficult to access. Back in
early 2015, npm Inc created a
private internal registry API that is
accessible exclusively to the npm website and npm CLI, and there are
no signs of that API becoming
publicly available any time soon.
Installation
npm install all-the-packages --save
When you install this package, a postinstall
script downloads
the npm registry metadata to a local JSON file, which is about 540 MB.
To refresh the module once you've already installed it, just reinstall:
npm i all-the-packages@latest
Usage
This package provides a simple event emitter that emits two events:
package
and end
.
const registry = require('all-the-packages')
registry
.on('package', function (pkg) {
console.log(`${pkg.name} - ${pkg.description}\n`)
})
.on('end', function () {
})
To get cleaner package data, use nice-package:
const registry = require('all-the-packages')
const NicePackage = require('nice-package')
const coolPackages = []
registry
.on('package', function (pkg) {
if (pkg.name.match('cool') || pkg.description.match('cool')) {
coolPackages.push(new NicePackage(pkg))
}
})
.on('end', function () {
})
Why Streams?
Streams are cool and all, but the main reason this module has a streaming
interface is RAM. It would be nice to just require
the module and have a
giant registry object to work with, but a 400 MB JSON file is too big to fit in
memory on most computers.
See Also
- package-stream: An endless stream of clean package data from the npm registry.
- nice-package: Clean up messy package metadata from the npm registry
Tests
npm install
npm test
Dependencies
- JSONStream: rawStream.pipe(JSONStream.parse()).pipe(streamOfObjects)
- event-stream: construct pipes of streams of events
Dev Dependencies
- tap-spec: Formatted TAP output like Mocha's spec reporter
- tape: tap-producing test harness for node and browsers
License
MIT
Generated by package-json-to-readme