readdirp

What is readdirp?

The readdirp npm package is a Node.js module that allows for reading the contents of directories recursively with a stream API. It provides a flexible and powerful way to filter, map, and reduce directory contents in an efficient manner. This package is particularly useful for tasks that involve file system operations, such as building file trees, searching for files with specific patterns, or processing files in batches.

What are readdirp's main functionalities?

Streaming directory contents

This feature allows you to stream the contents of a directory, filtering for specific file types (in this case, JavaScript files). It's useful for processing files as they are found.

const readdirp = require('readdirp');

readdirp('.', { fileFilter: '*.js' })
  .on('data', (entry) => {
    console.log(entry.path);
  })
  .on('end', () => console.log('Done'));

Promise API for directory reading

This feature provides a Promise-based API for reading directories, allowing for asynchronous file processing with better error handling and integration with async/await syntax.

const readdirp = require('readdirp');

readdirp('.', { depth: 1, fileFilter: '*.js' })
  .then(files => {
    files.forEach(file => console.log(file.path));
  })
  .catch(error => console.error('Error:', error));

Custom filter and entry formatting

This feature demonstrates how to use custom filters for file selection and how to format the entries returned by readdirp. It's useful for more complex file selection criteria and custom output formatting.

const readdirp = require('readdirp');

const options = {
  fileFilter: (entry) => entry.basename.startsWith('test'),
  entryType: 'files',
  alwaysStat: true,
  depth: 2
};

readdirp('.', options)
  .on('data', (entry) => {
    console.log(`${entry.path} - ${entry.stats.size} bytes`);
  });

Other packages similar to readdirp

glob

The 'glob' package provides pattern matching functionality to select files in directories. It's similar to readdirp in that it can be used to search for files, but it uses glob patterns instead of providing a stream API or Promise-based interface.

fs-extra

The 'fs-extra' package extends the built-in Node.js 'fs' module with additional file system methods, including recursive directory reading. While it offers similar functionality to readdirp, fs-extra provides a broader set of file system operations, making it more of a general-purpose library.

walk

The 'walk' package is another Node.js module for recursively reading directory contents. It is similar to readdirp but focuses more on event-based directory walking. Compared to readdirp, 'walk' might offer a simpler API for some use cases but lacks the advanced filtering and mapping capabilities.

README

readdirp

Recursive version of fs.readdir.

var readdirp = require('readdirp'); 

readdirp({ root: './test/bed', fileFilter: '*.js' }, function (err, res) {
    // do something with JavaScript files and all directories
});

Meant to be one of the recursive versions of fs functions, e.g., like mkdirp.

Table of Contents generated with DocToc

• Installation
• API
  • options
  • callbacks
    • allProcessed
    • fileProcessed
  • entry info
  • Filters
• More Examples

Installation

npm install readdirp

API

readdirp (options, callback1 [, callback2])

Reads given root recursively and returns list of files and directories with stats attached.

options

• root: path in which to start reading and recursing into subdirectories

• fileFilter: filter to include/exclude files found (see Filters for more)

• directoryFilter: filter to include/exclude directories found and to recurse into (see Filters for more)

• depth: depth at which to stop recursing even if more subdirectories are found

callbacks

If callback2 is given, callback1 functions as the fileProcessed callback, and callback2 as the allProcessed callback.

If only callback1 is given, it functions as the allProcessed callback.

allProcessed

• function with err and res parameters, e.g., function (err, res) { ... }
• err: array of errors that occurred during the operation, res may still be present, even if errors occurred
• res: collection of file/directory entry infos

fileProcessed

• function with entry info parameter e.g., function (entryInfo) { ... }

entry info

Has the following properties:

• parentDir : directory in which entry was found (relative to given root)

• fullParentDir : full path to parent directory

• name : name of the file/directory

• path : path to the file/directory (relative to given root)

• fullPath : full path to the file/directory found

• stat : built in stat object

• Example: (assuming root was /User/dev/readdirp)

    parentDir     :  'test/bed/root_dir1',
    fullParentDir :  '/User/dev/readdirp/test/bed/root_dir1',
    name          :  'root_dir1_subdir1',
    path          :  'test/bed/root_dir1/root_dir1_subdir1',
    fullPath      :  '/User/dev/readdirp/test/bed/root_dir1/root_dir1_subdir1',
    stat          :  [ ... ]
                
  

Filters

There are three different ways to specify filters for files and directories respectively.

• function: a function that takes an entry info as a parameter and returns true to include or false to exclude the entry

• glob string: a string (e.g., *.js) which is matched using minimatch, so go there for more information.

  Globstars (**) are not supported since specifiying a recursive pattern for an already recursive function doesn't make sense.

  Negated globs (as explained in the minimatch documentation) are allowed, e.g., !*.txt matches everything but text files.

• array of glob strings: either need to be all inclusive or all exclusive (negated) patterns otherwise an error is thrown.

  [ '*.json', '*.js' ] includes all JavaScript and Json files.

  [ '!.git', '!node_modules' ] includes all directories except the '.git' and 'node_modules'.

Directories that do not pass a filter will not be recursed into.

More Examples

var readdirp = require('readdirp');

// Glob file filter
readdirp({ root: './test/bed', fileFilter: '*.js' }, function (err, res) {
  // do something with JavaScript files and all directories
});

// Combined glob file filters
readdirp({ root: './test/bed', fileFilter: [ '*.js', '*.json' ] }, function (err, res) {
  // do something with JavaScript and Json files and all directories
});

// Combined negated directory filters
readdirp({ root: './test/bed', directoryFilter: [ '!.git', '!*modules' ] }, function (err, res) {
  // do something with all files and directories found outside '.git' or any modules directory 
});

// Function directory filter
readdirp(
  { root: './test/bed', directoryFilter: function (di) { return di.name.length === 9; } }, 
  function (err, res) {
  // do something with all files and directories found inside or matching directories whose name has length 9
})

// Limiting depth
readdirp({ root: './test/bed', depth: 1 }, function (err, res) {
  // do something with all files and directories found up to 1 subdirectory deep
});

// Using file processed callback
readdirp(
    { root: '.' }
  , function(fileInfo) { 
      // do something with file here
    } 
  , function (err, res) {
      // all done, move on or do final step for all files and directories here
    }
);

For more examples see the readdirp tests