libcloudfs

libcloudfs.

Motivation

// make it work, make it simple, make is fast, make it clean

Unlimited drive.

libcloudfs is cloudfs main storage driver. libcloudfs is a file system that stores all its data in the cloud. it stores file contents in a CAS designed cloud object storage backend openstack swift and files metadata (inode table) in an SQLlite database.

Installation

libcloudfs is a storage driver, mostly, please refer to the cloudfs project for installation instructions.

Project structure

• The libcloudfs main driver
• An isolated inode management API (see sqlfs)
• A fuse bindings interface
• A battle tested casfs backend, to challenge implementation, confirm design and stress
• An openstack/swift driver

Roadmap

• Read Only POC using full openstack creds
• Full Read Only POC using tempUrl keys (no full creds ever required)
• Writable Inodes POC (rename, delete, mkdir)
• Minimal fs driver
• Initial test flow (through fs driver)
• Proper deployment flow
• Writable/editable files (fs mode)
• a bit better test suite (win/linux)
• (create dedicated project for writable big fs chunks) - see casfs - dedicated project
• Testable SeqWriteHTTP module
• Integrated writable files (with bigfile support)
• Publish read-only mode
• Temp write file consolidation

Upcoming roadmap

• garbage collector

Features

• Simple by design
• Available on all platforms (linux & Windows)
• Unlimited file size (cloudfs is mostly designed to store and manage 100k files of 10GB+ - aka HD BR rips)
• Fast (sqlite is actually fastest than most file system)
• large subset of POSIX including reading/writing files, directories, rename, symlinks, mode, uid/gid, and extended attributes
• renames do not invole any kind of server side copy
• native file deduplication - through CAS
• Compatible with existing CAS

Additional features

• nice configuration GUI
• Directroy tree snapshot / rollback / sealing (pure SQL)
• Instant file deletion (pure SQL)
• Server side TAR creation (so content duplication) - through static large object.

Related

• casfs main backend
• sqlfs inode backend
• s3ql python based, non CAS (but fixed block)

Credits/thanks

• 131 - author
• fuse bindings