@zenfs/core

ZenFS

ZenFS is an in-browser file system that emulates the Node JS file system API and supports storing and retrieving files from various backends. ZenFS also integrates nicely into the Emscripten file system.

Backends

ZenFS is highly extensible, and includes many builtin filesystem backends:

• InMemory: Stores files in-memory. It is a temporary file store that clears when the user navigates away.
• OverlayFS: Mount a read-only file system as read-write by overlaying a writable file system on top of it. Like Docker's overlayfs, it will only write changed files to the writable file system.
• AsyncMirror: Use an asynchronous backend synchronously. Invaluable for Emscripten; let your Emscripten applications write to larger file stores with no additional effort!
  • AsyncMirror loads the entire contents of the async file system into a synchronous backend during construction. It performs operations synchronous file system and then queues them to be mirrored onto the asynchronous backend.
• FolderAdapter: Wraps a file system, and scopes all interactions to a subfolder of that file system.

More backends can be defined by separate libraries, so long as they extend they implement ZenFS.FileSystem. Multiple backends can be active at once at different locations in the directory hierarchy.

ZenFS supports a number of other backends (as @zenfs/fs-[name]).

For more information, see the API documentation for ZenFS.

Installing

npm install @zenfs/core

Building

• Make sure you have Node and NPM installed. You must have Node v18 or newer.
• Install dependencies with npm install
• Build using npm run build
• You can find the built code in dist.

Usage

🛈 The examples are written in ESM. If you are using CJS, you can require the package. If running in a borwser you can add a script tag to your HTML pointing to the browser.min.js and use ZenFS via the global ZenFS object.

import { fs } from '@zenfs/core';

fs.writeFileSync('/test.txt', 'Cool, I can do this in the browser!');

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

Using different backends

A InMemory backend is created by default. If you would like to use a different one, you must configure ZenFS. It is recommended to do so using the configure function. Here is an example using the LocalStorage backend from @zenfs/fs-dom:

import { configure, fs } from '@zenfs/core';
import '@zenfs/fs-dom'; // size effects are needed

// you can also add a callback as the last parameter instead of using promises
await configure({ fs: 'LocalStorage' });

if (!fs.existsSync('/test.txt')) {
	fs.writeFileSync('/test.txt', 'This will persist across reloads!');
}

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

Using multiple backends

You can use multiple backends by passing an object to configure which maps paths to file systems. The following example mounts a zip file to /zip, in-memory storage to /tmp, and IndexedDB browser-local storage to /home (note that / has the default in-memory backend):

import { configure } from '@zenfs/core';
import '@zenfs/fs-dom';
import '@zenfs/fs-zip';
import Buffer from 'buffer';

const zipData = await (await fetch('mydata.zip')).arrayBuffer();

await configure({
	'/mnt/zip': {
		fs: 'ZipFS',
		options: {
			zipData: Buffer.from(zipData)
		}
	},
	'/tmp': 'InMemory',
	'/home': 'IndexedDB',
};

FS Promises API

The FS promises API is exposed as promises.

import { configure, promises } from '@zenfs/core';
import '@zenfs/fs-dom';

await configure({ '/': 'IndexedDB' });

const exists = await promises.exists('/myfile.txt');
if (!exists) {
	await promises.write('/myfile.txt', 'Lots of persistant data');
}

ZenFS does not provide a seperate method for importing promises in its built form. If you are using Typescript, you can import the promises API from source code (perhaps to reduce you bundle size). Doing so it not recommended as the files may be moved without notice.

Using asynchronous backends synchronously

You may have noticed that attempting to use a synchronous method on an asynchronous backend (e.g. IndexedDB) results in a "not supplied" error (ENOTSUP). If you wish to use an asynchronous backend synchronously you need to wrap it in an AsyncMirror:

import { configure, fs } from '@zenfs/core';
import '@zenfs/fs-dom';

await configure({
	'/': { fs: 'AsyncMirror', options: { sync: { fs: 'InMemory' }, async: { fs: 'IndexedDB' } } }
});

fs.writeFileSync('/persistant.txt', 'My persistant data'); // This fails if you configure the FS as IndexedDB

Advanced usage

Creating backends

If you would like to create backends without configure, you may do so by importing the backend's class and calling its Create method. You can import the backend directly or with backends:

import { configure, backends, InMemory } from '@zenfs/core';

console.log(backends.InMemory === InMemory) // they are the same

const inMemoryFS = await InMemory.Create();

⚠ Instances of backends follow the internal ZenFS API. You should never use a backend's method unless you are extending a backend.

Coming soon:

import { configure, InMemory } from '@zenfs/core';

const inMemoryFS = new InMemory();
await inMemoryFS.whenReady();

Mounting

If you would like to mount and unmount backends, you can do so using the mount and umount functions:

import { fs, InMemory } from '@zenfs/core';

const inMemoryFS = await InMemory.Create(); // create an FS instance

fs.mount('/tmp', inMemoryFS); // mount

fs.umount('/tmp'); // unmount /tmp

This could be used in the "multiple backends" example like so:

import { configure, fs, ZipFS } from '@zenfs/core';
import '@zenfs/fs-dom';
import '@zenfs/fs-zip';
import Buffer from 'buffer';

await configure({
	'/tmp': 'InMemory',
	'/home': 'IndexedDB',
};

fs.mkdirSync('/mnt');

const res = await fetch('mydata.zip');
const zipData = Buffer.from(await res.arrayBuffer());
const zipFs = await ZipFS.Create({ zipData });
fs.mount('/mnt/zip', zipFs);

// do stuff with the mounted zip

fs.umount('/mnt/zip'); // finished using the zip

Using with bundlers

ZenFS exports a drop-in for Node's fs module (up to the version of @types/node in package.json), so you can use it for your bundler of preference using the default export.

ESBuild

tsconfig.json

{
	...
	"paths": {
		"fs": ["node_modules/zenfs/dist/index.js"]
	}
	...
}

Why tsconfig.json?

Webpack:

module.exports = {
	// ...
	resolve: {
		alias: {
			fs: require.resolve('zenfs'),
		},
	},
	// ...
};

Rollup:

import alias from '@rollup/plugin-alias';

export default {
	// ...
	plugins: [
		alias({
			entries: [{ find: 'fs', replacement: 'zenfs' }],
		}),
	],
	// ...
};

Using with Emscripten

You can use any synchronous ZenFS file systems with Emscripten.

import { EmscriptenFS } from '@zenfs/fs-emscripten';
const BFS = new EmscriptenFS(); // Create a ZenFS Emscripten FS plugin.
FS.createFolder(FS.root, 'data', true, true); // Create the folder that we'll turn into a mount point.
FS.mount(BFS, { root: '/' }, '/data'); // Mount BFS's root folder into the '/data' folder.

If you want to use an asynchronous backend, you must wrap it in an AsyncMirror.

Testing

Run unit tests with npm test.

Citing

ZenFS is a component of the Doppio and Browsix research projects from the PLASMA lab at the University of Massachusetts Amherst. If you decide to use ZenFS in a project that leads to a publication, please cite the academic papers on Doppio and Browsix:

John Vilk and Emery D. Berger. Doppio: Breaking the Browser Language Barrier. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (2014), pp. 508–518.

@inproceedings{VilkDoppio,
	author		= {John Vilk and
							 Emery D. Berger},
	title		 = {{Doppio: Breaking the Browser Language Barrier}},
	booktitle = {Proceedings of the 35th {ACM} {SIGPLAN} Conference on Programming Language Design and Implementation},
	pages		 = {508--518},
	year			= {2014},
	url			 = {http://doi.acm.org/10.1145/2594291.2594293},
	doi			 = {10.1145/2594291.2594293}
}

Bobby Powers, John Vilk, and Emery D. Berger. Browsix: Bridging the Gap Between Unix and the Browser. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (2017), pp. 253–266.

@inproceedings{PowersBrowsix,
	author		= {Bobby Powers and
							 John Vilk and
							 Emery D. Berger},
	title		 = {{Browsix: Bridging the Gap Between Unix and the Browser}},
	booktitle = {Proceedings of the Twenty-Second International Conference on Architectural
							 Support for Programming Languages and Operating Systems},
	pages		 = {253--266},
	year			= {2017},
	url			 = {http://doi.acm.org/10.1145/3037697.3037727},
	doi			 = {10.1145/3037697.3037727}
}

License

ZenFS is licensed under the MIT License. See LICENSE for details.