@zenfs/core

ZenFS

ZenFS is an in-browser file system that emulates the Node JS file system API and supports storing and retrieving files from various backends. ZenFS also integrates nicely with other tools.

Backends

ZenFS is highly extensible, and includes many builtin filesystem backends:

• InMemory: Stores files in-memory. It is a temporary file store that clears when the user navigates away.
• OverlayFS: Mount a read-only file system as read-write by overlaying a writable file system on top of it. Like Docker's overlayfs, it will only write changed files to the writable file system.
• AsyncMirror: Use an asynchronous backend synchronously. Invaluable for Emscripten; let your Emscripten applications write to larger file stores with no additional effort!
  • AsyncMirror loads the entire contents of the async file system into a synchronous backend during construction. It performs operations synchronous file system and then queues them to be mirrored onto the asynchronous backend.
• FolderAdapter: Wraps a file system, and scopes all interactions to a subfolder of that file system.

More backends can be defined by separate libraries, so long as they extend they implement ZenFS.FileSystem. Multiple backends can be active at once at different locations in the directory hierarchy.

ZenFS supports a number of other backends (as @zenfs/fs-[name]).

For more information, see the API documentation for ZenFS.

Installing

npm install @zenfs/core

Building

• Make sure you have Node and NPM installed. You must have Node v18 or newer.
• Install dependencies with npm install
• Build using npm run build
• You can find the built code in dist.

Usage

🛈 The examples are written in ESM. If you are using CJS, you can require the package. If running in a browser you can add a script tag to your HTML pointing to the browser.min.js and use ZenFS via the global ZenFS object.

import fs from '@zenfs/core';

fs.writeFileSync('/test.txt', 'Cool, I can do this in the browser!');

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

Using different backends

A InMemory backend is created by default. If you would like to use a different one, you must configure ZenFS. It is recommended to do so using the configure function. Here is an example using the Storage backend from @zenfs/fs-dom:

import { configure, fs, registerBackend } from '@zenfs/core';
import { StorageFileSystem } from '@zenfs/fs-dom';
registerBackend(StorageFileSystem);

// you can also add a callback as the last parameter instead of using promises
await configure({ fs: 'Storage' });

if (!fs.existsSync('/test.txt')) {
	fs.writeFileSync('/test.txt', 'This will persist across reloads!');
}

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

Using multiple backends

You can use multiple backends by passing an object to configure which maps paths to file systems. The following example mounts a zip file to /zip, in-memory storage to /tmp, and IndexedDB storage to /home (note that / has the default in-memory backend):

import { configure, registerBackend } from '@zenfs/core';
import { IndexedDBFileSystem } from '@zenfs/fs-dom';
import { ZipFS } from '@zenfs/fs-zip';
import Buffer from 'buffer';
registerBackend(IndexedDBFileSystem, ZipFS);

const zipData = await (await fetch('mydata.zip')).arrayBuffer();

await configure({
	'/mnt/zip': {
		fs: 'ZipFS',
		options: {
			zipData: Buffer.from(zipData)
		}
	},
	'/tmp': 'InMemory',
	'/home': 'IndexedDB',
};

FS Promises API

The FS promises API is exposed as promises.

import { configure, promises, registerBackend } from '@zenfs/core';
import { IndexedDBFileSystem } from '@zenfs/fs-dom';
registerBackend(IndexedDBFileSystem);

await configure({ '/': 'IndexedDB' });

const exists = await promises.exists('/myfile.txt');
if (!exists) {
	await promises.write('/myfile.txt', 'Lots of persistant data');
}

ZenFS does not provide a seperate method for importing promises in its built form. If you are using Typescript, you can import the promises API from source code (perhaps to reduce you bundle size). Doing so it not recommended as the files may be moved without notice.

Using asynchronous backends synchronously

You may have noticed that attempting to use a synchronous method on an asynchronous backend (e.g. IndexedDB) results in a "not supplied" error (ENOTSUP). If you wish to use an asynchronous backend synchronously you need to wrap it in an AsyncMirror:

import { configure, fs } from '@zenfs/core';
import { IndexedDBFileSystem } from '@zenfs/fs-dom';
registerBackend(IndexedDBFileSystem);

await configure({
	'/': { fs: 'AsyncMirror', options: { sync: { fs: 'InMemory' }, async: { fs: 'IndexedDB' } } }
});

fs.writeFileSync('/persistant.txt', 'My persistant data'); // This fails if you configure the FS as IndexedDB

Advanced usage

Creating backends

If you would like to create backends without configure, you may do so by importing the backend's class and calling its Create method. You can import the backend directly or with backends:

import { configure, backends, InMemory } from '@zenfs/core';

console.log(backends.InMemory === InMemory) // they are the same

const inMemoryFS = await InMemory.Create();

⚠ Instances of backends follow the internal ZenFS API. You should never use a backend's method unless you are extending a backend.

Coming soon:

import { configure, InMemory } from '@zenfs/core';

const inMemoryFS = new InMemory();
await inMemoryFS.whenReady();

Mounting

If you would like to mount and unmount backends, you can do so using the mount and umount functions:

import { fs, InMemory } from '@zenfs/core';

const inMemoryFS = await InMemory.Create(); // create an FS instance

fs.mount('/tmp', inMemoryFS); // mount

fs.umount('/tmp'); // unmount /tmp

This could be used in the "multiple backends" example like so:

import { IndexedDBFileSystem } from '@zenfs/fs-dom';
import { ZipFS } from '@zenfs/fs-zip';
import Buffer from 'buffer';
registerBackend(IndexedDBFileSystem);

await configure({
	'/tmp': 'InMemory',
	'/home': 'IndexedDB',
};

fs.mkdirSync('/mnt');

const res = await fetch('mydata.zip');
const zipData = Buffer.from(await res.arrayBuffer());
const zipFs = await ZipFS.Create({ zipData });
fs.mount('/mnt/zip', zipFs);

// do stuff with the mounted zip

fs.umount('/mnt/zip'); // finished using the zip

Using with bundlers

ZenFS exports a drop-in for Node's fs module (up to the version of @types/node in package.json), so you can use it for your bundler of preference using the default export.

ESBuild

tsconfig.json

{
	...
	"paths": {
		"fs": ["node_modules/zenfs/dist/index.js"]
	}
	...
}

Why tsconfig.json?

Webpack:

module.exports = {
	// ...
	resolve: {
		alias: {
			fs: require.resolve('zenfs'),
		},
	},
	// ...
};

Rollup:

import alias from '@rollup/plugin-alias';

export default {
	// ...
	plugins: [
		alias({
			entries: [{ find: 'fs', replacement: 'zenfs' }],
		}),
	],
	// ...
};

Using with Emscripten

You can use any synchronous ZenFS file systems with Emscripten.

import { EmscriptenFSPlugin } from '@zenfs/fs-emscripten';
const BFS = new EmscriptenFSPlugin(); // Create a ZenFS Emscripten FS plugin.
FS.createFolder(FS.root, 'data', true, true); // Create the folder to turn into a mount point.
FS.mount(BFS, { root: '/' }, '/data'); // Mount BFS's root folder into /data.

If you want to use an asynchronous backend, you must wrap it in an AsyncMirror.

Testing

Run unit tests with npm test.

License

ZenFS is licensed under the MIT License.