cloudfs

Cloudfs.

// make it work, make it simple, make is fast, make it clean

Motivation

Unlimited drive.

cloudfs is a file system that stores all its data in the cloud. cloudfs store file contents in a CAS designed cloud object storage backend openstack swift and files metadata (inode table) in an SQL database (sqlite - see dedicated sqlfs project).

Project structure

The cloudfs is designed around simplicity.

• The inode management API (see sqlfs)
• A fuse bindings interface
• A battle tested casfs backend, to challenge implementation, confirm design and stress
• An openstack/swift driver

Roadmap

• Read Only POC using full openstack creds

• Full Read Only POC using tempUrl keys (no full creds ever required)

• Writable Inodes POC (rename, delete, mkdir)

• Minimal fs driver

• Initial test flow (through fs driver)

• Proper deployment flow

• Writable/editable files (fs mode)

• a bit better test suite (win/linux)

• (create dedicated project for writable big fs chunks) - see casfs - dedicated project

• Testable SeqWriteHTTP module <= current

• Integrated writable files (with bigfile support)

• Embbed configuration/web browse server

• Publish read-only mode

• With full test suite (e.g. winfsp/secfs test suite)

Background daemon & pending tasks

• Garbage collection
• Support for nwjs (fuse-binding & sqlite3)

Features

• Simple by design
• Unlimited file size (cloudfs is mostly designed to store and manage 100k files of 8GB+ - aka HD BR rips)
• Available on all platforms (linux & Windows)
• Fast (sqlite is actually fastest than most file system)
• large subset of POSIX including reading/writing files, directories, rename, symlinks, mode, uid/gid, and extended attributes
• renames do not invole any kind of server side copy
• native file deduplication - through CAS
• Compatible with existing CAS

Additional features

• nice configuration GUI
• Directroy tree snapshot / rollback / sealing (pure SQL)
• Instant file deletion (pure SQL)
• Server side TAR creation (so content duplication) - through static large object.

Related

• casfs main backend
• s3ql python based, non CAS (but fixed block)

Credits/thanks

• 131 - author
• fuse bindings