Product
Introducing Ruby Support in Socket
Socket is launching Ruby support for all users. Enhance your Rails projects with AI-powered security scans for vulnerabilities and supply chain threats. Now in Beta!
The fdir npm package is a fast and efficient directory crawler that allows you to read and list files in a directory tree. It is designed to be highly performant and can handle large directories with ease.
Basic Directory Crawling
This feature allows you to crawl a directory and get a list of all files with their full paths. The code sample demonstrates how to use fdir to crawl a directory and log the list of files.
const { fdir } = require('fdir');
const api = new fdir().withFullPaths().crawl('path/to/directory');
api.withPromise().then(files => console.log(files));
Filtering Files
This feature allows you to filter the files based on a condition. The code sample shows how to filter files to include only JavaScript files.
const { fdir } = require('fdir');
const api = new fdir().withFullPaths().filter(path => path.endsWith('.js')).crawl('path/to/directory');
api.withPromise().then(files => console.log(files));
Using Streams
This feature allows you to use streams to handle the files as they are found. The code sample demonstrates how to use a stream to log each file as it is crawled.
const { fdir } = require('fdir');
const api = new fdir().withFullPaths().crawl('path/to/directory');
api.withStream().on('data', file => console.log(file));
Using Async/Await
This feature allows you to use async/await syntax for better readability and control over asynchronous operations. The code sample shows how to use async/await to crawl a directory and log the list of files.
const { fdir } = require('fdir');
(async () => {
const api = new fdir().withFullPaths().crawl('path/to/directory');
const files = await api.withPromise();
console.log(files);
})();
fast-glob is a library for searching files by glob patterns. It is highly performant and supports advanced features like negative globs and multiple patterns. Compared to fdir, fast-glob is more focused on pattern matching and offers a rich set of options for globbing.
readdirp is a recursive version of fs.readdir with additional features like filtering and streaming. It is similar to fdir in that it allows you to read directories recursively, but it is less focused on performance and more on providing a simple API for directory traversal.
klaw is a file system walker that uses streams to traverse directories. It is similar to fdir in its use of streams but is generally slower. klaw is more suitable for applications where streaming is a primary requirement.
⚡ The Fastest: Nothing similar (in the NodeJS world) beats fdir
in speed. It can easily crawl a directory containing 1 million files in < 1 second.
💡 Stupidly Easy: fdir
uses expressive Builder pattern to build the crawler increasing code readability.
🤖 Zero Dependencies: fdir
only uses NodeJS fs
& path
modules.
🕺 Astonishingly Small: < 2KB in size gzipped & minified.
🔥 All Node Versions Supported: Unlike other similar libraries that have dropped support for Node versions < 10, fdir
supports all versions >= 6.
🖮 Hackable: Extending fdir
is extremely simple now that the new Builder API is here. Feel free to experiment around.
Do you like this project? Support me by donating, creating an issue, becoming a stargazer or opening a pull request. Thanks.
You can install using npm
:
$ npm i fdir
or Yarn:
$ yarn add fdir
const fdir = require("fdir").default;
// create the builder
const api = new fdir().withFullPaths().crawl("path/to/dir");
// get all files in a directory synchronously
const files = api.sync();
// or asynchronously
api.withPromise().then((files) => {
// do something with the result here.
});
I have written an in-depth documentation here.
Specs:
Notes:
Some people asked that I benchmark no-op
(without options) version of fdir
. I did and found no performance difference. The results were identical. (I didn't include it here as it wasn't anything special.)
Some other people were doubtful about the authenticity of these results due to frequency scaling, process overload, disk warmup etc. So I have updated the benchmark with new results that should resolve all those doubts. Here's the process I followed:
$ sudo cpupower frequency-set --governor performance
Last updated: May 13, 2020 (fdir v3.3.0)
$ yarn bench:glob
glob pattern used:
**.js
&**/**.js
Synchronous | Asynchronous |
---|---|
Last updated: May 10, 2020 (fdir v3.0.0)
$ yarn bench
Synchronous | Asynchronous |
---|---|
Older versions of fdir (1.x & 2.x) used synchronous lstat
call (lstatSync
) in the asynchronous API to acheive speed on Node < 10. This has been fixed in fdir 3.0.0.
Synchronous | |
---|---|
Copyright (c) 2020 Abdullah Atta under MIT. Read full text here.
FAQs
The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s
The npm package fdir receives a total of 1,977,249 weekly downloads. As such, fdir popularity was classified as popular.
We found that fdir demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket is launching Ruby support for all users. Enhance your Rails projects with AI-powered security scans for vulnerabilities and supply chain threats. Now in Beta!
Product
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
Product
We're launching a new set of license analysis and compliance features for analyzing, managing, and complying with licenses across a range of supported languages and ecosystems.