Socket
Book a DemoInstallSign in
Socket

@julesl23/s5js

Package Overview
Dependencies
Maintainers
1
Versions
5
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

@julesl23/s5js

Enhanced TypeScript SDK for S5 decentralized storage with path-based API, media processing, and directory utilities

latest
Source
npmnpm
Version
0.9.0-beta
Version published
Weekly downloads
2
-60%
Maintainers
1
Weekly downloads
ย 
Created
Source

Enhanced S5.js SDK

An enhanced JavaScript/TypeScript SDK for the S5 decentralized storage network, featuring a simple path-based API for file and directory operations.

Features

  • ๐Ÿš€ Path-based API: Simple get(), put(), delete(), list() operations
  • ๐Ÿ“ Directory Support: Full directory tree management with recursive operations
  • ๐Ÿ”„ Cursor Pagination: Efficient handling of large directories
  • ๐Ÿ” Built-in Encryption: Automatic encryption for private data
  • ๐Ÿ“ฆ CBOR Serialization: Deterministic encoding for cross-platform compatibility
  • ๐ŸŒ Browser & Node.js: Works in both environments
  • ๐Ÿ—‚๏ธ HAMT Sharding: Automatic directory sharding for millions of entries
  • ๐Ÿšถ Directory Walker: Recursive traversal with filters and resumable cursors
  • ๐Ÿ“‹ Batch Operations: High-level copy/delete operations with progress tracking
  • ๐Ÿ–ผ๏ธ Media Processing: WASM-based image metadata extraction with Canvas fallback
  • ๐ŸŽจ Color Analysis: Dominant color extraction and palette generation
  • ๐Ÿ“Š Bundle Optimization: Code-splitting support (~70KB gzipped total)
  • โœ… Real S5 Portal Integration: Fully tested with s5.vup.cx portal

Key Components

Core API

  • S5: Main client class for connection and identity management
  • FS5: File system operations with path-based API
  • S5UserIdentity: User identity and authentication

Utility Classes

  • DirectoryWalker: Recursive directory traversal with cursor support
  • BatchOperations: High-level copy/delete operations with progress tracking

Media Processing

  • MediaProcessor: Unified image metadata extraction with WASM/Canvas
  • BrowserCompat: Browser capability detection and strategy selection
  • CanvasMetadataExtractor: Fallback image processing using Canvas API

See the API Documentation for detailed usage examples.

Installation

The enhanced path-based API features are currently in development as part of a Sia Foundation grant project.

Prerequisites:

  • Node.js v20+ (for development and testing)
  • Python 3 (required for browser tests - used to run local HTTP server)
  • npm (comes with Node.js)

For production use:

npm install @s5-dev/s5js

To try the enhanced features:

# Clone the repository
git clone https://github.com/julesl23/s5.js
cd s5.js

# Install dependencies (includes TypeScript)
npm install

# Build the project
npm run build

# Run tests with real S5 portal
npm test

Status: These features are pending review and have not been merged into the main S5.js repository.

Quick Start

import { S5 } from "./dist/src/index.js";

// Create S5 instance and connect to real S5 portal
const s5 = await S5.create({
  initialPeers: [
    "wss://z2DWuPbL5pweybXnEB618pMnV58ECj2VPDNfVGm3tFqBvjF@s5.ninja/s5/p2p",
  ],
});

// Generate a new seed phrase (save this securely!)
const seedPhrase = s5.generateSeedPhrase();
console.log("Your seed phrase:", seedPhrase);

// Or recover from existing seed phrase
// const seedPhrase = "your saved twelve word seed phrase here";

await s5.recoverIdentityFromSeedPhrase(seedPhrase);

// Register on S5 portal (s5.vup.cx supports the new API)
await s5.registerOnNewPortal("https://s5.vup.cx");

// Initialize filesystem (creates home and archive directories)
await s5.fs.ensureIdentityInitialized();

// Store data
await s5.fs.put("home/documents/hello.txt", "Hello, S5!");

// Retrieve data
const content = await s5.fs.get("home/documents/hello.txt");
console.log(content); // "Hello, S5!"

// List directory contents
for await (const item of s5.fs.list("home/documents")) {
  console.log(`${item.type}: ${item.name}`);
}

Advanced Usage

import { DirectoryWalker, BatchOperations, MediaProcessor } from "./dist/src/index.js";

// Recursive directory traversal
const walker = new DirectoryWalker(s5.fs, '/');
for await (const entry of walker.walk("home", { maxDepth: 3 })) {
  console.log(`${entry.path} (${entry.type})`);
}

// Batch operations with progress
const batch = new BatchOperations(s5.fs);
const result = await batch.copyDirectory("home/source", "home/backup", {
  onProgress: (progress) => {
    console.log(`Copied ${progress.processed} items...`);
  }
});
console.log(`Completed: ${result.success} success, ${result.failed} failed`);

// Media processing - extract image metadata
await MediaProcessor.initialize();
const imageBlob = await fetch('/path/to/image.jpg').then(r => r.blob());
const metadata = await MediaProcessor.extractMetadata(imageBlob);
console.log(`Image: ${metadata.width}x${metadata.height} ${metadata.format}`);
console.log(`Dominant colors:`, metadata.dominantColors);

Testing with Real S5 Portal

The enhanced S5.js has been successfully integrated with real S5 portal infrastructure. To test:

This test creates a new identity and verifies all functionality:

node test/integration/test-fresh-s5.js

Expected output: 100% success rate (9/9 tests passing)

2. Full Integration Test

Comprehensive test of all features:

node test/integration/test-s5-full-integration.js

3. Direct Portal API Test

Tests direct portal communication:

node test/integration/test-portal-direct.js

4. Batch Operations Test

Tests BatchOperations (copy/delete) with real S5 portal:

node test/integration/test-batch-real.js

This test validates:

  • Copy directory with progress tracking
  • Delete directory with progress tracking
  • Error handling modes
  • Metadata preservation

5. Media Extensions Test (Phase 6.3)

Tests FS5 media integration (putImage, getThumbnail, getImageMetadata, createImageGallery) with real S5 instance:

node test/integration/test-media-real.js

This test validates:

  • Image upload with automatic thumbnail generation
  • Metadata extraction (format, dimensions)
  • Thumbnail retrieval (pre-generated and on-demand)
  • Gallery creation with manifest.json
  • Directory integration with media operations
  • Path-based API (no CID exposure)

Expected output: 10/10 tests passing

Important Notes

  • Use Fresh Identities: The new deterministic key derivation system requires fresh identities. Old accounts created with the previous system won't work.
  • Portal URL: Use https://s5.vup.cx which has the updated API. Other portals may not have the required updates.
  • Path Requirements: All paths must start with either home/ or archive/

Performance Benchmarks

The enhanced S5.js includes comprehensive performance benchmarks to verify HAMT efficiency and scaling behaviour.

Running Benchmarks

Local Mock Benchmarks (Fast)

Test HAMT performance with mock S5 API:

# Basic HAMT verification
node test/mocked/integration/test-hamt-local-simple.js

# Comprehensive scaling test (up to 100K entries)
node test/mocked/integration/test-hamt-mock-comprehensive.js

Real Portal Benchmarks (Network)

Test with actual S5 portal (requires internet connection):

# Minimal real portal test
node test/integration/test-hamt-real-minimal.js

# HAMT activation threshold test
node test/integration/test-hamt-activation-real.js

# Full portal performance analysis
node test/integration/test-hamt-real-portal.js

Benchmark Results

See BENCHMARKS.md for detailed performance analysis showing:

  • HAMT activation at exactly 1000 entries
  • O(log n) scaling verified up to 100K+ entries
  • ~800ms per operation on real S5 network
  • Memory usage of ~650 bytes per entry

For production deployments, these benchmarks confirm the implementation is ready for large-scale directory operations.

Bundle Size & Code Splitting

The library supports multiple import strategies to optimize bundle size:

// Full bundle (~60KB compressed with brotli)
import { S5, MediaProcessor } from "s5";

// Core only - no media features (~60KB compressed)
import { S5, FS5 } from "s5/core";

// Media only - for lazy loading (~10KB compressed)
import { MediaProcessor } from "s5/media";

// Advanced CID API - for power users (~60KB compressed)
import { FS5Advanced, formatCID, parseCID } from "s5/advanced";

// Dynamic import for code-splitting
const { MediaProcessor } = await import("s5/media");

Monitor bundle sizes with:

npm run analyze-bundle

Advanced CID API

For power users who need direct access to Content Identifiers (CIDs), the Advanced API provides content-addressed storage capabilities without affecting the simplicity of the path-based API.

When to Use

Use the Advanced API if you:

  • Need to reference content by its cryptographic hash
  • Are building content-addressed storage applications
  • Require deduplication or content verification
  • Work with distributed systems that use CIDs

Use the Path-based API if you:

  • Need simple file storage (most use cases)
  • Prefer traditional file system operations
  • Want paths to be more meaningful than hashes

Quick Example

import { S5 } from "s5";
import { FS5Advanced, formatCID, parseCID } from "s5/advanced";

// Setup
const s5 = await S5.create();
await s5.recoverIdentityFromSeedPhrase(seedPhrase);
const advanced = new FS5Advanced(s5.fs);

// Store data and get CID
await s5.fs.put('home/document.txt', 'Important data');
const cid = await advanced.pathToCID('home/document.txt');
console.log(`CID: ${formatCID(cid, 'base32')}`);

// Share the CID string
const cidString = formatCID(cid, 'base58btc');

// Recipient: retrieve by CID alone
const receivedCID = parseCID(cidString);
const data = await advanced.getByCID(receivedCID);
console.log(data); // "Important data"

// Find path from CID
const path = await advanced.cidToPath(receivedCID);
console.log(path); // "home/document.txt"

Available Methods

FS5Advanced Class (4 essential methods):

  • pathToCID(path) - Extract CID from file/directory path
  • cidToPath(cid) - Find path for a given CID
  • getByCID(cid) - Retrieve data by CID directly
  • putByCID(data) - Store content-only and return CID

Composition Pattern:

  • For path + CID: Use fs.put(path, data) then advanced.pathToCID(path)
  • For metadata + CID: Use fs.getMetadata(path) then advanced.pathToCID(path)

CID Utilities:

  • formatCID(cid, encoding?) - Format CID as multibase string
  • parseCID(cidString) - Parse CID from string
  • verifyCID(cid, data, crypto) - Verify CID matches data
  • cidToString(cid) - Convert to hex string

See the Advanced API Documentation for complete details.

Encryption

Enhanced S5.js includes built-in encryption using XChaCha20-Poly1305, providing both confidentiality and integrity for sensitive data.

Basic Encryption

// Auto-generate encryption key
await s5.fs.put("home/secrets/credentials.json", sensitiveData, {
  encryption: {
    algorithm: "xchacha20-poly1305",
  },
});

// Retrieve and decrypt automatically
const data = await s5.fs.get("home/secrets/credentials.json");
console.log(data); // Original decrypted data

User-Provided Encryption Keys

// Use your own 32-byte encryption key
const myKey = new Uint8Array(32); // Your secure key
crypto.getRandomValues(myKey);

await s5.fs.put("home/private/document.txt", "Secret content", {
  encryption: {
    algorithm: "xchacha20-poly1305",
    key: myKey, // Use specific key
  },
});

// Decryption uses key from metadata automatically
const content = await s5.fs.get("home/private/document.txt");

Features

  • Algorithm: XChaCha20-Poly1305 (AEAD cipher)
  • Key Size: 256-bit (32 bytes)
  • Chunk-based: Large files encrypted in 256 KiB chunks
  • Transparent: Automatic encryption/decryption
  • Secure: Each chunk uses unique nonce

Security Considerations

โš ๏ธ Important: Encryption keys are stored in directory metadata. Anyone with directory read access can decrypt files. This design provides:

  • โœ… Convenience: No separate key management needed
  • โœ… Automatic decryption with directory access
  • โš ๏ธ Access control: Secure your directory access credentials

For complete encryption documentation, examples, and security best practices, see the Encryption section in API.md.

Documentation

Development

This is an enhanced version of s5.js being developed under an 8-month grant from the Sia Foundation. The project implements a new format using:

  • New Format: CBOR serialization with DirV1 specification (replaces MessagePack)
  • Path-based API: Simple file operations with familiar syntax
  • HAMT sharding: Automatic directory sharding for efficient large directory support
  • Directory utilities: Recursive operations with progress tracking and error handling
  • Deterministic Key Derivation: Subdirectory keys derived from parent keys
  • Real Portal Integration: Successfully tested with s5.vup.cx

Note: This is a clean implementation that does NOT maintain backward compatibility with old S5 data formats.

Building

npm run build     # Compile TypeScript
npm run dev       # Watch mode
npm run test      # Run tests

Development Commands

npm run build       # Compile TypeScript to JavaScript
npm run dev         # Watch mode for development
npm run type-check  # Run TypeScript type checking

Testing

npm run test        # Run real implementation tests only
npm run test:run    # Run tests once
npm run test:mocked # Run mock-based tests
npm run test:all    # Run all tests (real + mocked)
npm run test:ui     # Run tests with UI
npm run test:coverage # Generate coverage report

# Run specific test suites
npm run test:run test/fs/cid-utils.test.ts test/fs/fs5-advanced.test.ts  # Advanced CID API unit tests (74 tests)

Test Organization

  • test/ - Real implementation tests using actual S5.js functionality

    • Run with npm test (30+ test files, 284+ tests)
    • Tests core functionality without mocks
  • test/mocked/ - Mock-based unit and performance tests

    • Run with npm run test:mocked (15 test files)
    • Includes HAMT performance benchmarks and isolated component tests
    • test/mocked/integration/ - Mock-based integration and performance tests
  • test/integration/ - Real S5 integration tests with actual network connections

    • Tests that connect to real S5 portals (e.g., s5.vup.cx)
    • Use real seed phrases and portal registration

Running Real S5 Portal Integration Tests

For comprehensive testing with real S5 infrastructure, use the standalone integration test scripts:

# Build the project first
npm run build

# Run Advanced CID API integration tests with real S5 portal
node test/integration/test-advanced-cid-real.js

Note: These tests:

  • Connect to real S5 portals (default: https://s5.vup.cx)
  • Use actual registry operations with 5+ second propagation delays
  • Run sequentially to avoid registry conflicts
  • Generate temporary test files (auto-cleaned)
  • Take ~2 minutes to complete (18 tests)

Media Processing Tests & Demos

Phase 5 Media Processing Foundation

The media processing implementation includes comprehensive demos and tests. All Phase 5 deliverables are complete with 100% test coverage.

Quick Start - Run All Demos

# Build the project first
npm run build

# Run all Node.js demos
node demos/media/benchmark-media.js         # Performance benchmarking
node demos/media/demo-pipeline.js           # Pipeline initialization
node demos/media/demo-metadata.js           # Metadata extraction
node demos/media/test-media-integration.js  # Integration tests (Node.js)

# Run browser tests (all 20 tests pass in browser)
./demos/media/run-browser-tests.sh          # Linux/Mac
# Windows: npx http-server -p 8080, then open http://localhost:8080/demos/media/browser-tests.html

# View code-splitting demo (requires HTTP server)
# Linux/Mac: ./demos/media/run-browser-tests.sh (uses port 8081)
# Windows: npx http-server -p 8081, then open http://localhost:8081/demos/media/demo-splitting-simple.html

โš™๏ธ Platform-Specific Notes

Node.js Test Expectations:

When running node demos/media/test-media-integration.js:

  • โœ… Expected: 17/20 tests pass (85%)
  • โŒ 3 tests fail due to Node.js platform limitations (NOT bugs):
    • "WASM Module Loading" - Canvas is 42x faster in Node.js, WASM not loaded (correct)
    • "Process Real JPEG Image - Width" - Node.js lacks full Canvas API for dimensions (works in browser)
    • "Dominant Color Extraction" - Node.js can't access pixel data (works in browser)

Browser Test Expectations:

  • โœ… All 20/20 tests pass (100%)

Windows Users:

The bash script ./demos/media/run-browser-tests.sh won't work in Windows CMD. Use one of these alternatives:

# Option 1: Using npx (recommended - no Python needed)
npx http-server -p 8080

# Option 2: Using Python (if installed)
python -m http.server 8080

# Then open in browser:
http://localhost:8080/demos/media/browser-tests.html

Linux/Mac Users:

# Use the provided script
./demos/media/run-browser-tests.sh

# Automatically opens: http://localhost:8081/demos/media/browser-tests.html

๐Ÿงช Browser Tests - All 20 Tests Passing

Expected Results:

  • โœ… 20/20 tests pass in browser (100%)
  • โœ… Full WASM functionality
  • โœ… Real dimensions, color extraction, all features working

Tests Include:

  • MediaProcessor initialization
  • Browser capability detection
  • Strategy selection (wasm-worker, canvas-main, etc.)
  • PNG/JPEG/GIF/BMP/WebP metadata extraction
  • Dominant color extraction
  • Transparency detection
  • Aspect ratio calculation
  • Processing time tracking
  • Speed classification (fast/normal/slow)
  • WASM to Canvas fallback
  • Invalid image handling
  • Timeout support
  • Orientation detection
  • Concurrent extractions
  • WASM module validation
  • Multiple format support

Evidence Column: Each test shows verification data proving it passes

๐Ÿ“Š Performance Benchmarking

Run: node demos/media/benchmark-media.js

Output:

  • Processes test images with WASM and Canvas strategies
  • Generates performance comparison table
  • Saves baseline metrics to baseline-performance.json
  • Shows processing times, memory usage, success rates

Expected Results:

  • Canvas faster in Node.js (175x faster due to no Web Workers)
  • WASM initialization: ~83ms first image, <1ms subsequent
  • Canvas: consistent 0.03-0.31ms
  • Strategy adapts to environment (canvas-main for Node.js)

๐Ÿ”ง Pipeline Setup Demo

Run: node demos/media/demo-pipeline.js

Demonstrates:

  • Environment capability detection
  • Smart strategy selection based on capabilities
  • WASM module initialization with progress tracking
  • Memory management and cleanup
  • Fallback handling scenarios

Key Features:

  • Shows decision tree for strategy selection
  • ASCII pipeline flow diagram
  • Real-time progress tracking
  • Memory delta measurements

๐ŸŽจ Metadata Extraction

Run: node demos/media/demo-metadata.js

Processes:

  • All image formats (PNG, JPEG, GIF, BMP, WebP)
  • Magic byte format detection
  • Processing speed classification
  • Generates HTML report at metadata-report.html

Note: In Node.js, dimensions show 0x0 (expected limitation). Works fully in browser.

๐Ÿ“ฆ Code-Splitting Demo

Prerequisites: Requires HTTP server

Windows:

npx http-server -p 8081
# Then open: http://localhost:8081/demos/media/demo-splitting-simple.html

Linux/Mac:

./demos/media/run-browser-tests.sh
# Then open: http://localhost:8081/demos/media/demo-splitting-simple.html

Shows:

  • Core bundle: 195 KB (-27% from full)
  • Media bundle: 79 KB (loaded on-demand)
  • Real image processing with loaded modules
  • Bundle size comparison table
  • Live implementation examples

Expected Test Results

Browser Environment (Full Support):

  • โœ… 20/20 tests passing
  • โœ… Real image dimensions extracted
  • โœ… Dominant colors working
  • โœ… WASM module loads
  • โœ… Web Workers available
  • โœ… Strategy: wasm-worker

Node.js Environment (Limited Canvas):

  • โœ… 16-19/20 tests passing (expected)
  • โš ๏ธ Dimensions show 0x0 for some formats (no full Canvas API)
  • โš ๏ธ No color extraction (needs pixel access)
  • โœ… Format detection works
  • โœ… Falls back to canvas-main strategy
  • โœ… All operations < 50ms (fast)

Why These Results Are Expected

  • Node.js Limitations: No Web Workers, limited Canvas API, so it uses fallbacks
  • Browser Full Support: All features work with real Canvas and WASM
  • Adaptive Strategy: System detects capabilities and chooses optimal path
  • Performance: Canvas faster in Node.js, WASM better for larger images in browser

Media Processing API Usage

import { MediaProcessor } from 's5/media';

// Initialize (automatic in browser)
await MediaProcessor.initialize();

// Extract metadata
const blob = new Blob([imageData], { type: 'image/png' });
const metadata = await MediaProcessor.extractMetadata(blob);

console.log(`Image: ${metadata.width}x${metadata.height}`);
console.log(`Format: ${metadata.format}`);
console.log(`Processing: ${metadata.processingTime}ms`);

Test Server

For integration testing with mock S5 services:

node test/mocked/integration/test-server.js  # Start mock server on port 3000

See test-server-README.md for details.

Project Architecture

Technology Stack

  • Language: TypeScript (ES2022 target, ESNext modules)
  • Runtime: Dual-targeted for Browser and Node.js
  • Test Framework: Vitest with global test functions
  • Crypto: @noble libraries for cryptographic operations
  • Storage: IndexedDB (browser) and memory-level (Node.js)
  • Serialization: CBOR via cbor-x
  • Networking: WebSocket-based P2P connections

Module Structure

  • src/api/ - Core S5 API interfaces and crypto implementations
  • src/fs/ - File system operations (FS5 implementation)
    • dirv1/ - CBOR-based directory format implementation
    • hamt/ - Hash Array Mapped Trie for large directories
    • utils/ - Directory walker and batch operations
  • src/media/ - Media processing and metadata extraction
    • wasm/ - WebAssembly module wrapper for image processing
    • fallback/ - Canvas-based fallback implementation
    • compat/ - Browser compatibility detection
  • src/identity/ - User identity and authentication
  • src/node/ - P2P networking and registry operations
  • src/kv/ - Key-value storage abstractions
  • src/encryption/ - Encryption utilities
  • src/identifier/ - Content identifiers and multibase encoding
  • src/util/ - Utility functions
  • src/exports/ - Modular export paths for code-splitting

Project Status

  • โœ… Month 1: Project Setup - Complete
  • โœ… Month 2: Path Helpers v0.1 - Complete
  • โœ… Month 3: Path-cascade Optimization & HAMT - Complete
  • โœ… Month 4: Directory Utilities - Complete
  • โœ… Month 5: Media Processing Foundation - Complete
  • โœ… Month 6: Advanced Media Processing - Complete
  • โœ… S5 Portal Integration - Complete (100% test success rate)
  • โœ… Phase 6.5: Advanced CID API - Complete (74 tests passing)
  • โœ… Month 7: Testing & Performance - Substantially Complete (~85%)
  • ๐Ÿšง Month 8: Documentation & Upstream Integration - In Progress (~40%)

See MILESTONES.md for detailed progress.

Grant Milestone 5 Deliverables

Milestone 5 (Advanced Media Processing) has been completed and validated. All grant requirements have been met and exceeded:

Requirements Met โœ…

  • Thumbnail Generation โœ…

    • JPEG, PNG, and WebP format support
    • Smart cropping with face/object detection
    • Size constraints: All thumbnails โ‰ค64 KB (average: 29.5 KB)
    • 21 dedicated tests passing
  • Progressive Rendering โœ…

    • Three strategies implemented: Blur, Scan Lines, Interlaced
    • Browser compatibility with graceful fallbacks
    • Visual demo validated in Chrome, Edge, and Firefox
    • 27 dedicated tests passing
  • Browser Compatibility Matrix โœ…

    • Tested: Chrome 90+, Firefox 88+, Edge 90+, Node.js 20+
    • 10 capability detection features (Canvas, WebP, WASM, etc.)
    • Graceful fallback system implemented
    • 31 browser compatibility tests passing
  • Bundle Size Optimization โœ…

    • Requirement: โ‰ค700 KB (compressed)
    • Achieved: 60.09 KB (brotli) - 10x under budget
    • Modular exports for code-splitting: s5, s5/core, s5/media, s5/advanced

Documentation & Validation

For complete evidence and testing instructions, see:

  • MILESTONE5_EVIDENCE.md - Comprehensive evidence document with:

    • Detailed proof of all requirements met
    • Test results (437 tests passing, 225+ media-specific)
    • Browser compatibility matrix
    • Performance metrics and bundle analysis
    • Integration test results on real S5 network
  • MILESTONE5_TESTING_GUIDE.md - Step-by-step validation guide with:

    • How to run unit tests (npm run test:run)
    • How to run integration test (node test/integration/test-media-real.js)
    • How to launch browser demo (./test/browser/run-demo.sh)
    • Bundle size verification steps
    • Troubleshooting guide

Quick Validation

# 1. Run unit tests (437 tests)
npm run test:run

# 2. Run integration test with real S5 network
npm run build
node test/integration/test-media-real.js

# 3. Launch progressive rendering browser demo
./test/browser/run-demo.sh

# 4. Verify bundle size
npm run build
brotli -f -k dist/src/index.js
du -h dist/src/index.js.br  # Should show ~60 KB

Status: All Milestone 5 deliverables complete and ready for review.

Completed Phases โœ…

  • Phase 1: Core Infrastructure (CBOR, DirV1 types)
  • Phase 2: Path-Based API (get, put, delete, list, getMetadata)
  • Phase 3: HAMT Integration (auto-sharding at 1000+ entries)
  • Phase 4: Directory Utilities (walker, batch operations)
  • Phase 5: Media Processing Foundation (WASM + Canvas with browser detection)
  • Phase 6: Advanced Media Processing (thumbnail generation, progressive loading, FS5 integration, bundle optimization)
  • Phase 6.5: Advanced CID API (74 tests passing, s5/advanced export)
  • Phase 7: Testing & Performance (280+ tests, benchmarks complete)

Remaining Work โณ

  • Phase 8: Documentation & Upstream Integration
    • Community outreach (blog post, forum announcements)
    • Upstream PR to s5-dev/s5.js
    • Optional: Firefox/Safari browser testing

Performance

The implementation has been benchmarked to ensure efficient operation:

  • HAMT activation: Automatic at 1000+ entries
  • Scaling: O(log n) performance verified up to 100K+ entries
  • Memory usage: ~650 bytes per directory entry
  • Network latency: ~800ms per operation on real S5 network

See BENCHMARKS.md for detailed results.

Testing & Integration

  • For S5 portal testing, see the test files mentioned above
  • For integration testing with external services, see test-server-README.md

Troubleshooting

"Invalid base length" errors

  • Solution: Use a fresh seed phrase. Old accounts have incompatible key structures.

Directory not found errors

  • Solution: Ensure you call ensureIdentityInitialized() after portal registration
  • All paths must start with home/ or archive/

Portal connection issues

  • Use https://s5.vup.cx which has the updated API
  • Ensure you have Node.js v20+ for proper crypto support

Important Notes

  • Format: Uses new CBOR/DirV1 format - NOT compatible with old S5 data
  • Paths: Must start with home/ or archive/
  • Portal: Use https://s5.vup.cx for testing (has updated API)
  • Identity: Requires fresh seed phrases (old accounts incompatible)

Contributing

This project is being developed under a Sia Foundation grant. For contributions or issues, please refer to the grant proposal.

License

Licensed under either of:

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

This is an enhanced version of s5.js being developed under an 8-month grant from the Sia Foundation. The project implements a new format using CBOR serialization with the DirV1 specification.

Keywords

s5

FAQs

Package last updated on 31 Oct 2025

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts