fs-tree-diff

What is fs-tree-diff?

The fs-tree-diff npm package is used to efficiently compare and manage differences between two file system trees. It is particularly useful for tasks that involve synchronizing directories, detecting changes, and applying updates based on those changes.

What are fs-tree-diff's main functionalities?

Compare two directories

This feature allows you to compare two directories and generate a patch that represents the differences between them. The code sample demonstrates how to create two file system trees and calculate the patch.

const FSTree = require('fs-tree-diff');
const entriesA = FSTree.fromEntries([{ relativePath: 'file1.txt', mode: 0o100644, size: 123, mtime: new Date() }]);
const entriesB = FSTree.fromEntries([{ relativePath: 'file2.txt', mode: 0o100644, size: 456, mtime: new Date() }]);
const treeA = new FSTree({ entries: entriesA });
const treeB = new FSTree({ entries: entriesB });
const patch = treeA.calculatePatch(treeB);
console.log(patch);

Apply a patch to a directory

This feature allows you to apply a patch to a directory, effectively synchronizing it with another directory. The code sample demonstrates how to calculate a patch and then apply it to a file system tree.

const FSTree = require('fs-tree-diff');
const entriesA = FSTree.fromEntries([{ relativePath: 'file1.txt', mode: 0o100644, size: 123, mtime: new Date() }]);
const entriesB = FSTree.fromEntries([{ relativePath: 'file2.txt', mode: 0o100644, size: 456, mtime: new Date() }]);
const treeA = new FSTree({ entries: entriesA });
const treeB = new FSTree({ entries: entriesB });
const patch = treeA.calculatePatch(treeB);
FSTree.applyPatch(treeA, patch);
console.log(treeA.entries);

Generate entries from a directory

This feature allows you to generate entries from a directory, which can then be used to create a file system tree. The code sample demonstrates how to read a directory and generate entries for each file.

const FSTree = require('fs-tree-diff');
const fs = require('fs');
const path = require('path');
function generateEntries(dir) {
  const entries = [];
  fs.readdirSync(dir).forEach(file => {
    const filePath = path.join(dir, file);
    const stats = fs.statSync(filePath);
    entries.push({ relativePath: file, mode: stats.mode, size: stats.size, mtime: stats.mtime });
  });
  return entries;
}
const entries = generateEntries('./my-directory');
console.log(entries);

Other packages similar to fs-tree-diff

diff

The 'diff' package provides a way to compare text differences between two strings or files. While it focuses on text comparison, fs-tree-diff is more specialized in comparing file system trees and managing directory synchronization.

rsync

The 'rsync' package is a utility for efficiently transferring and synchronizing files across computer systems. It is more comprehensive and includes network transfer capabilities, whereas fs-tree-diff is focused on local file system tree comparisons.

chokidar

The 'chokidar' package is a file watcher that tracks changes in the file system and triggers events. While it can detect changes, it does not provide the same level of detailed comparison and patching capabilities as fs-tree-diff.

README

fs-tree-diff

FSTree provides the means to calculate a patch (set of operations) between one file system tree and another.

The possible operations are:

• unlink – remove the specified file
• rmdir – remove the specified folder
• mkdir – create the specified folder
• create – create the specified file
• change – update the specified file to reflect changes

The operations chosen aim to minimize the amount of IO required to apply a given patch. For example, a naive rm -rf of a directory tree is actually quite costly, as child directories must be recursively traversed, entries stated.. etc, all to figure out what first must be deleted. Since we patch from tree to tree, discovering new files is both wasteful and un-needed.

The operations will also be provided in a correct order, allowing us to safely replay operations without having to first confirm the FS is as we expect. For example, unlinks for files will occur before a rmdir of those files' parent dir. Although the ordering will be safe, a specific order is not guaranteed.

A simple example:

var FSTree = require('fs-tree-diff');
var current = FSTree.fromPaths([
  'a.js'
]);

var next = FSTree.fromPaths([
  'b.js'
]);

current.calculatePatch(next) === [
  ['unlink', 'a.js'],
  ['create', 'b.js']
];

A slightly more complicated example:

var FSTree = require('fs-tree-diff');
var current = FSTree.fromPaths([
  'a.js',
  'b/',
  'b/f.js'
]);

var next = FSTree.fromPaths([
  'b.js',
  'b/',
  'b/c/',
  'b/c/d.js',
  'b/e.js'
]);

current.calculatePatch(next) === [
  ['unlink', 'a.js', entryA],
  ['create', 'b.js', entryB],
  ['mkdir', 'b/c', entryBC],
  ['create', 'b/c/d.js', entryBCD],
  ['create', 'b/e.js', entryBE]
  ['unlink', 'b/f.js', entryBF],
]

Now, the above examples do not demonstrate update operations. This is because when providing only paths, we do not have sufficient information to check if one entry is merely different from another with the same relativePath.

For this, FSTree supports more complex input structure. To demonstrate, We will use the walk-sync module. Which provides higher fidelity input, allowing FSTree to also detect changes. More on what an entry from walkSync.entries is

var walkSync = require('walk-sync');

// path/to/root/foo.js
// path/to/root/bar.js
var current = new FSTree({
  entries: walkSync.entries('path/to/root')
});

writeFileSync('path/to/root/foo.js', 'new content');
writeFileSync('path/to/root/baz.js', 'new file');

var next = new FSTree({
  entries: walkSync.entries('path/to/root')
});

current.calculatePatch(next) === [
  ['update', 'foo.js', entryFoo], // mtime + size changed, so this input is stale and needs updating.
  ['create', 'baz.js', entryBaz]  // new file, so we should create it
  /* bar stays the same and is left inert*/
];

The entry objects provided depend on the operation. For rmdir and unlink operations, the current entry is provided. For mkdir, change and create operations the new entry is provided.

API

The public API is:

• FSTree.fromPaths initialize a tree from an array of string paths.

• FSTree.fromEntries initialize a tree from an object containing an entries property. Each entry must have the following properties (but may have more):

  • relativePath
  • mode
  • size
  • mtime

• FSTree.prototype.calculatePatch(newTree, isEqual) calculate a patch against newTree. Optionally specify a custom isEqual (see Change Calculation).

Input

FSTree.fromPaths and FSTree.fromEntries both validate their inputs. Inputs must be sorted, path-unique (ie two entries with the same relativePath but different sizes would still be illegal input) and include intermediate directories.

For example, the following input is invaild

FSTree.fromPaths([
  // => missing a/ and a/b/
  'a/b/c.js'
]);

To have FSTree sort and expand (include intermediate directories) for you, add the option sortAndExpand).

FStree.fromPaths([
	'a/b/q/r/bar.js',
	'a/b/c/d/foo.js',
], { sortAndExpand: true });

// The above is equivalent to

FSTree.fromPaths([
	'a/',
	'a/b/',
	'a/b/c/',
	'a/b/c/d/',
	'a/b/c/d/foo.js',
	'a/b/q/',
	'a/b/q/r/',
	'a/b/q/r/bar.js',
]);

Entry

FSTree.fromEntries requires you to supply your own Entry objects. Your entry objects must contain the following properties:

• relativePath
• mode
• size
• mtime

They must also implement the following API:

• isDirectory() true iff this entry is a directory

Change Calculation

When a prior entry has a relativePath that matches that of a current entry, a change operation is included if the new entry is different from the previous entry. This is determined by calling isEqual, the optional second argument to calculatePatch. If no isEqual is provided, a default isEqual is used.

The default isEqual treats directories as always equal and files as different if any of the following properties have changed.

• mode
• size
• mtime

User specified isEqual will often want to use the default isEqual, so it is exported on FSTree.

Example

var defaultIsEqual = FSTtreeDiff.isEqual;

function isEqualCheckingMeta(a, b) {
  return defaultIsEqual(a, b) && isMetaEqual(a, b);
}

function isMetaEqual(a, b) {
  // ...
}

Changelog

v0.5.0

• [BREAKING] drop unlinkdir and linkDir as operations. Downstream can implement this by examining entries, eg by marking them beforehand as broccoli-merge-trees does.
• [BREAKING] unlink and rmdir operations are now passed the entry
• [BREAKING] entries must be lexigraphicaly sorted by relative path. To do this implicitly, use sortAndExpand.
• [BREAKING] entries must include intermediate directories. To do this implicitly, use sortAndExpand.
• reworked implementation to diff via two sorted arrays
• performance improvements
• return entires as-provided, preserving user-specified metadata
• directories in patches always end with a trailing slash
• fixes various issues related to directory state transitions
• directories can now receive change patches if user-supplied meta has property changes