readdir-cluster

readdir-cluster

Create a cluster of workers to iterate through the filesystem

Usage

Complete API Documentation.

import readdirCluster, { Stat } from 'readdir-cluster'

// note that Stat is not the same as fs.Stats as it has functions removed, as it needed to be serialisable
function iterator(path: string, filename: string, stat: Stat) {
    // skip directories and files that start with .
    if (filename[0] === '.') return false
    // do not recurse into directories
    if (stat.directory) return false
}

const paths = await readdirCluster({ directory: '.', iterator })
console.log(paths)

Performance

Benchmarks:

• Running readdir-cluster . returns 7388 files in 500ms

• Running readdir with recursive: true returns 7388 files in 100ms

  import { readdir } from 'fs'
  readdir('.', { recursive: true }, (err, files) => {
      if (err) console.error(err)
      else if (files.length) process.stdout.write(files.join('\n') + '\n')
  })
  

• Running fdir returns 6480 files in 100ms

  import { fdir } from 'fdir'
  const api = new fdir().withBasePath().crawl(process.argv[2])
  api.withPromise().then((files) => {
      if (files.length) process.stdout.write(files.join('\n') + '\n')
  })
  

Recommendations:

• if you target Node.js 18.7 and above, you should use fs.readdir with recursive: true
• if you target older Node.js versions, you should use @bevry/fs-list which uses fs-readdir if available otherwise uses the find executable which is even more performant (60ms instead of 100ms)
• if you target older Node.js versions and you want a stat object, use readdir-cluster
• if you target Nodejs 12 and above, and want a lot of customisation, use fdir

As for why this package exists, readdir-cluster was created in 2005, recursive was added to Node.js in 2023, and fdir was created in 2020. That said, there are several issues that could potentially improve readdir-cluster performance.

Install

npm

Install Globally

• Install: npm install --global readdir-cluster
• Executable: readdir-cluster

Install Locally

• Install: npm install --save readdir-cluster
• Executable: npx readdir-cluster
• Import: import pkg from ('readdir-cluster')
• Require: const pkg = require('readdir-cluster').default

Editions

This package is published with the following editions:

• readdir-cluster aliases readdir-cluster/index.cjs which uses the Editions Autoloader to automatically select the correct edition for the consumer's environment
• readdir-cluster/source/index.ts is TypeScript source code with Import for modules
• readdir-cluster/edition-es2022/index.js is TypeScript compiled against ES2022 for Node.js 14 || 16 || 18 || 20 || 21 with Require for modules
• readdir-cluster/edition-es2017/index.js is TypeScript compiled against ES2017 for Node.js 8 || 10 || 12 || 14 || 16 || 18 || 20 || 21 with Require for modules
• readdir-cluster/edition-es2015/index.js is TypeScript compiled against ES2015 for Node.js 6 || 8 || 10 || 12 || 14 || 16 || 18 || 20 || 21 with Require for modules
• readdir-cluster/edition-es5/index.js is TypeScript compiled against ES5 for Node.js 4 || 6 || 8 || 10 || 12 || 14 || 16 || 18 || 20 || 21 with Require for modules
• readdir-cluster/edition-es2017-esm/index.js is TypeScript compiled against ES2017 for Node.js 12 || 14 || 16 || 18 || 20 || 21 with Import for modules
• readdir-cluster/edition-types/index.d.ts is TypeScript compiled Types with Import for modules

History

Discover the release history by heading on over to the HISTORY.md file.

Backers

Code

Discover how to contribute via the CONTRIBUTING.md file.

Authors

• Benjamin Lupton — Accelerating collaborative wisdom.

Maintainers

• Benjamin Lupton — Accelerating collaborative wisdom.

Contributors

• Benjamin Lupton — view contributions

Finances

Sponsors

• Andrew Nesbitt — Software engineer and researcher
• Balsa — We're Balsa, and we're building tools for builders.
• Codecov — Empower developers with tools to improve code quality and testing.
• Poonacha Medappa
• Rob Morris
• Sentry — Real-time crash reporting for your web apps, mobile apps, and games.
• Syntax — Syntax Podcast

Donors

• Andrew Nesbitt
• Armen Mkrtchian
• Balsa
• Chad
• Codecov
• dr.dimitru
• Elliott Ditman
• entroniq
• GitHub
• Hunter Beast
• Jean-Luc Geering
• Michael Duane Mooring
• Michael Harry Scepaniak
• Mohammed Shah
• Mr. Henry
• Nermal
• Pleo
• Poonacha Medappa
• Rob Morris
• Robert de Forest
• Sentry
• ServieJS
• Skunk Team
• Syntax
• WriterJohnBuck

License

Unless stated otherwise all works are:

• Copyright © Benjamin Lupton

and licensed under:

• Artistic License 2.0