fpkg

#fpkg (find npm package)

fpkg (short for find package), is a node command line program that searches the entire npm registry for all packages from terminal based on the given search term(s) and outputs the results to the console (or you can redirect the results to a file) as the stream of json data being processed on the fly. It makes a GET request to https://registry.npmjs.org/-/all to get the latest info about all packages. Then, JSONStream parses the stream of json data and event-stream maps it to relevant info of each package. fpkg doesn't do any indexing, caching or stuff like that. It tries to get the latest relevant data and deliver it to the user.

It is kind of important to know that fpkg does not use this uri https://registry.npmjs.org/-/_view/byKeyword?startkey=[%22keyword%22]&endkey=[%22keyword%22,{}]&group_level=3 because as far as I've learned (I may be wrong) this uri serves for the cases that you want given keyword(s) to be found only in the package keywords list. However, it is different than what fpkg tries to do, which is finding a match not only in the keywords but also in package names and descriptions.

###why

Well, because of issues with npm search regarding processing very large json file and process out of memory, like this, or this one, which looks more like a design thing than a bug though.

###install

use as a command line utility

Please run npm install -g fpkg. If you encounter with a problem or getting EACCES error, read fixing-npm-permissions.

use programmatically

run npm install fpkg

###config file

fpkg comes with a simple config .fpkgrc.json file that will be placed as a hidden file in the user home directory $HOME/.fpkgrc.json. It has two fields: verbose and highlight_color. This is what it looks like:

{
  "verbose": true,

  "highlight_color": {
  
    "red": false,
    "green": false,
    "yellow": false,
    "blue": false,
    "magenta": false,
    "cyan": false,
    "white": false,
    "black": false
  }
}

verbose is for verbose output, which means log raw output as it finds any relevant packages. If any packages found, you'll get a pretty, columnified result anyway.

gray is the default highlight_color; If you are ok with it, then leave the value of all colors as false. If you don't want gray, set any color that you want to true and that will overwrite the default value.

###command line usage

run fpkg <keyword>

example: fpkg express

example: fpkg parse json stream

Depending on the search term(s), the output may be very long. For instance, fpkg express returns over a hefty 6800 packages. Just keep in mind in these situations, it is usually useful to redirect the output to a file as well, in order to find what we were looking for easier.

output to console only (default)

fpkg <keyword>

output to both console and file

you can run something like,

fpkg <keyword> 2>&1 | tee result.log

or for multiple keywords,

fpkg <keyword0> <keyword1> <keyword2> 2>&1 | tee result.log

output to a file only (no output to the console)

fpkg keyword > result.log

###sample output

###api

fpkg.fetch()

make a GET request to https://registry.npmjs.org/-/all to fetch data for all npm packages and emit pkg event that sends the package info object after validating the received json stream data.

fpkg.emitter

an instance of the EventEmitter class. It emits only one event pkg that sends the package info object.

###api usage

You can retrieve all npm packages info as an object in a relatively fast manner

const fpkg = require('fpkg');

// make a request to the npm registry and process JSON stream events
fpkg.fetch();

// emits 'pkg' event when receives data from event stream.
// received data is an object containing the parsed npm package info

fpkg.emitter.on('pkg', (pkg_obj) => {
  console.log(pkg_obj);
});

you can get json like,

fpkg.emitter.on('pkg', (pkg_obj) => {
  console.log(JSON.stringify(pkg_obj));
});

to give you a better idea of what a typical obj for each package looks like,

{ name: ' ',
  description: ' ',
  'dist-tags': { latest: '0.0.4' },
  maintainers: [ { name: '', email: ' ' } ],
  author: { name: ' ', email: ' ' },
  repository:
   { type: 'git',
     url: 'https://github.com/...' },
  homepage: 'https://github.com/...',
  bugs: { url: 'https://github.com/...' },
  keywords: ['', ''],
  license: 'GPLv3',
  readmeFilename: 'README.md',
  time: { modified: '2015-08-27T12:13:24.974Z' },
  versions: { '0.0.4': 'latest' } }

Enjoy using fpkg.