walkdir

What is walkdir?

The walkdir npm package is a utility for recursively traversing directories and handling files and directories found within. It provides a simple and efficient way to walk through directory trees, making it useful for tasks such as file system analysis, batch processing of files, and more.

What are walkdir's main functionalities?

Basic Directory Traversal

This feature allows you to traverse a directory and log each file and directory found. The callback function receives the path and stat object for each item.

const walk = require('walkdir');

walk('/path/to/directory', function(path, stat) {
  console.log('Found:', path);
});

Filtering Files

This feature demonstrates how to filter files based on certain criteria, such as file extension. In this example, only JavaScript files are logged.

const walk = require('walkdir');

walk('/path/to/directory', function(path, stat) {
  if (stat.isFile() && path.endsWith('.js')) {
    console.log('Found JavaScript file:', path);
  }
});

Handling Errors

This feature shows how to handle errors that may occur during directory traversal. The 'error' event is emitted with the path and error details.

const walk = require('walkdir');

const emitter = walk('/path/to/directory');

emitter.on('file', function(path, stat) {
  console.log('Found file:', path);
});

emitter.on('error', function(path, err) {
  console.error('Error for path', path, ':', err);
});

Asynchronous Traversal

This feature demonstrates synchronous directory traversal, where the entire directory tree is read and returned as an array of paths.

const walk = require('walkdir');

const paths = walk.sync('/path/to/directory');

paths.forEach(function(path) {
  console.log('Found:', path);
});

Other packages similar to walkdir

fs-extra

fs-extra is a package that extends the native Node.js fs module with additional methods, including recursive directory traversal. It provides a higher-level API for file system operations, making it more versatile but also more complex compared to walkdir.

glob

glob is a package for matching files using patterns. It can be used to find files and directories that match specific patterns, making it useful for tasks like file searching and filtering. Unlike walkdir, glob focuses on pattern matching rather than directory traversal.

readdirp

readdirp is a recursive version of fs.readdir, providing a stream-based API for reading directories. It is similar to walkdir in that it allows for recursive directory traversal, but it uses streams for more efficient handling of large directory trees.

README

walkdir

Find files. Walks a directory tree emitting events based on what it finds. Presents a familliar callback/emitter/sync interface. Walk a tree of any depth. This is a performant option any pull requests to make it more so will be taken into consderation..

Example

var walk = require('walkdir');

//async with path callback 

walk('../', function(path, stat) {
  console.log('found: ', path);
});

//use async emitter to capture more events

var emitter = walk('../');

emitter.on('file', function(filename, stat) {
  console.log('file from emitter: ', filename);
});


//sync with callback

walk.sync('../', function(path, stat) {
  console.log('found sync:', path);
});

//sync just need paths

var paths = walk.sync('../');
console.log('found paths sync: ', paths);

// async await/promise!
let result = await walk.async('../',{return_object:true})
//result['path'] = {statObject}

install

npm install walkdir

arguments

walkdir(path, [options], [callback]) walkdir.sync(path, [options], [callback]);

• path

  • the starting point of your directory walk

• options. supported options are

  • general

{
  /**
  * follow symlinks. default FALSE
  */
  "follow_symlinks"?: boolean,
  /**
    * only go one level deep. convenience param.
    */ 
  "no_recurse"?: boolean,
  /**
    * only travel to max depth. emits an error if hit.
    */
  "max_depth"?: number,
  /**
    * on filesystems where inodes are not unique like windows (or perhaps hardlinks) some files may not be emitted due to inode collision.
    * turning off this behavior may be required but at the same time may lead to hitting max_depth via link loop.
    */
  "track_inodes"?: boolean;
  /**
    * make this syncronous. the same as calling walkdir.sync
    */
  "sync"?:boolean,
  /**
    * return an object of {path:stat} instead of just the resolved path names
    */
  "return_object"?: boolean,
  /**
    * dont build up an internal list or object of all of the paths. this can be an important optimization for listing HUGE trees.
    */
  "no_return"?: boolean,
  /**
    * filter. filter an array of paths from readdir
    */
  "filter"?:(directory:string,files:string[])=>string[]|Promise<string[]>,
  /**
    *  pass in a custom fs object like gracfeful-fs
    *  needs stat, lstat, readdir, readlink and sync verisons if you use sync:true
    */
  "fs"?:any,
  /*** 
   * default True. if false this will use stat insteqad of lstat and not find links at all.
   */
  "find_links?":boolean,
}

• walkdir.sync/walkdir.async only

  {
    "return_object": false, // if true the sync return will be in {path:stat} format instead of [path,path,...]
    "no_return": false, // if true null will be returned and no array or object will be created with found paths. useful for large listings
  }
  

• callback

  • this is bound to the path event of the emitter. its optional in all cases.

    callback(path, stat)
    

events

non error type events are emitted with (path,stat). stat is an instanceof fs.Stats

path

fired for everything

file

fired only for regular files

directory

fired only for directories

link

fired when a symbolic link is found

end

fired when the entire tree has been read and emitted.

socket

fired when a socket descriptor is found

fifo

fired when a fifo is found

characterdevice

fired when a character device is found

blockdevice

fired when a block device is found

targetdirectory

fired for the stat of the path you provided as the first argument. is is only fired if it is a directory.

empty

fired for empty directory

error events

error type events are emitted with (path,error). error being the error object returned from an fs call or other opperation.

error

if the target path cannot be read an error event is emitted. this is the only failure case.

fail

when stat or read fails on a path somewhere in the walk and it is not your target path you get a fail event instead of error. This is handy if you want to find places you dont have access too.

notes

the async emitter returned supports 3 methods

###end stop a walk in progress

###pause pause the walk. no more events will be emitted until resume

###resume resume the walk

ignore(path or array of paths)

will not traverse these directories. may be called in the path event handler to ignore dynamically.

var walk = require('walkdir');
var p = require('path');
walk('/', function(path, stat) {
  // ignore all .git directories.
  if (p.basename(path) === '.git') {
    this.ignore(path)
  }
})

cancel a walk in progress

//cancel a walk in progress within callback.

var walk = require('walkdir');
walk('../', function(path, stat) {
  this.end();
});

//cancel a walk in progress with emitter handle
var walk = require('walkdir');
var emitter = walk('../');

doSomethingAsync(function() {
  emitter.end();
})

thanks

thanks to substack. the interface for this module is based off of node-findit

contributing

see CONTRIBUTING.md for guidelines. this is an open opensource project.

Changelog

v0.3.0

feat: added find_links option to toggle between stat and lstat. defaults to existing behavior.