Product
Introducing Ruby Support in Socket
Socket is launching Ruby support for all users. Enhance your Rails projects with AI-powered security scans for vulnerabilities and supply chain threats. Now in Beta!
The fdir npm package is a fast and efficient directory crawler that allows you to read and list files in a directory tree. It is designed to be highly performant and can handle large directories with ease.
Basic Directory Crawling
This feature allows you to crawl a directory and get a list of all files with their full paths. The code sample demonstrates how to use fdir to crawl a directory and log the list of files.
const { fdir } = require('fdir');
const api = new fdir().withFullPaths().crawl('path/to/directory');
api.withPromise().then(files => console.log(files));
Filtering Files
This feature allows you to filter the files based on a condition. The code sample shows how to filter files to include only JavaScript files.
const { fdir } = require('fdir');
const api = new fdir().withFullPaths().filter(path => path.endsWith('.js')).crawl('path/to/directory');
api.withPromise().then(files => console.log(files));
Using Streams
This feature allows you to use streams to handle the files as they are found. The code sample demonstrates how to use a stream to log each file as it is crawled.
const { fdir } = require('fdir');
const api = new fdir().withFullPaths().crawl('path/to/directory');
api.withStream().on('data', file => console.log(file));
Using Async/Await
This feature allows you to use async/await syntax for better readability and control over asynchronous operations. The code sample shows how to use async/await to crawl a directory and log the list of files.
const { fdir } = require('fdir');
(async () => {
const api = new fdir().withFullPaths().crawl('path/to/directory');
const files = await api.withPromise();
console.log(files);
})();
fast-glob is a library for searching files by glob patterns. It is highly performant and supports advanced features like negative globs and multiple patterns. Compared to fdir, fast-glob is more focused on pattern matching and offers a rich set of options for globbing.
readdirp is a recursive version of fs.readdir with additional features like filtering and streaming. It is similar to fdir in that it allows you to read directories recursively, but it is less focused on performance and more on providing a simple API for directory traversal.
klaw is a file system walker that uses streams to traverse directories. It is similar to fdir in its use of streams but is generally slower. klaw is more suitable for applications where streaming is a primary requirement.
⚡ Extremely Fast: Nothing beats fdir
in speed. It can easily crawl a directory containing 10k files in about 13ms.
💡 Stupidly Easy: fdir
only has 2 functions; sync
and async
for crawling the file system synchronously or asynchronously.
🤖 Zero Dependencies: fdir
uses pure NodeJS fs
& path
namespaces. Nothing else is ever touched.
🕺 Astonishingly Small: Only 2KB in size. Can be used virtually anywhere.
You can install using npm
:
$ npm i --save fdir
or Yarn:
$ yarn add fdir
It makes no difference to me.
const fdir = require("fdir");
// get all files in a directory synchronously
const files = fdir.sync("path/to/dir");
// or asynchronously
fdir.async("path/to/dir").then(/*blah blah blah*/);
And that's it.
$ yarn benchmark
Specs:
Notes:
no-op
(without options) version of fdir
. I did and found no performance difference. The results were identical. (I didn't include it here as it wasn't anything special.)Synchronous (7386 files) | Asynchronous (7386 files) |
---|---|
Note: As latest version of rrdir
doesn't support Node < 8, I had to use version 2.0.0. Everything else is fully updated.
Synchronous (7386 files) | Asynchronous (7386 files) |
---|---|
fdir
is very small so there's not much to the API.
fdir.sync(string, Options): String[]
This is often the fastest way to get files. However, it will block the main "thread" so use it with caution with large directories.
fdir.async(string, Options): Promise<String[]>
Not always the fastest but works without blocking the street, so that's a plus.
Options
Ah, the options. Not many of them. At least not as many as I'd hoped for.
includeDirs: boolean
Whether to include directories in the array returned.
default: false
includeBasePath: boolean
Whether to include the base path for each file.
default: true
searchFn: Function
Use this to filter out files.
Example:
fdir.sync("node_modules", {
searchFn: path => path.includes(".git")
});
default: undefined
maxDepth: number
The max number of levels fdir
should crawl before stopping. The lower the faster.
default: undefined (i.e. infinity)
isExcludedDir: Function
A list of directories to exclude.
Note:
fdir
expects an Object not an array.
Example:
const isExcludedDir = path => path.includes(".bin");
fdir.sync("node_modules", { isExcludedDir });
default: undefined
And that's it.
1. I looked at the code and there's nothing special. How is it so damn fast then?
Well, that's the whole point. fdir
exists to prove to the "young" generation that you don't need to use special constructs or special methods to gain speed. Just a bit of patience and brains.
2. Why create this? What's the point?
I know you don't give a shit. Fine. There's no point behind this. It's "just for fun". No, wait. Actually, I created this, first of all, for me. I needed fast directory access in another app of mine, so fdir
came into being.
3. Why are all the other libraries so slow?
Because they did not spend enough time optimizing it. Most developers give readability and cool code more importance than actual performance and usability. I have seen a library claiming to be the fastest by inverting the benchmarks. Literally. Gave me quite the scare until I went and fixed the benchmark. It was actually one of the slowest. :O
4. How long did it take you to create this?
Ummm. Maybe 18 hours? Make it a day.
5. Are you looking for a job?
Am I? Well, are you offering a job? If yes, I am interested. :D
6. Why should I give a shit?
You shouldn't. But here's my email in case you do: thecodrr[at]protonmail.com. Don't worry, I don't bite.
Would love if you throw a coffee over here. Or just be, you know, polite and give me a star? Maybe even follow me?
Copyright (c) 2020 Abdullah Atta under MIT. Read full text here.
FAQs
The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s
The npm package fdir receives a total of 1,977,249 weekly downloads. As such, fdir popularity was classified as popular.
We found that fdir demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Socket is launching Ruby support for all users. Enhance your Rails projects with AI-powered security scans for vulnerabilities and supply chain threats. Now in Beta!
Product
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
Product
We're launching a new set of license analysis and compliance features for analyzing, managing, and complying with licenses across a range of supported languages and ecosystems.