@mohamedhabibwork/storagekit

storagekit

Unified TypeScript file management across multiple storage providers — with strongly-typed native provider options preserved instead of flattened into a lowest-common-denominator API.

import { createStorage } from '@mohamedhabibwork/storagekit';

const storage = await createStorage({
  type: 's3',
  bucket: 'uploads',
  region: 'eu-central-1',
});

await storage.upload('users/100/avatar.jpg', fileStream, {
  contentType: 'image/jpeg',
  // normalized, works on every provider
  cacheControl: 'public,max-age=31536000',
  // real AWS options, typed per storage type
  native: {
    StorageClass: 'INTELLIGENT_TIERING',
    ServerSideEncryption: 'AES256',
  },
});

Change type: 's3' to type: 'azure' and TypeScript now offers Azure-native options (tier: 'Cool', SAS conditions, …) — and rejects AWS ones.

Providers

Provider	Entrypoint	SDK
Local filesystem	storagekit/local	Node built-ins only
AWS S3	storagekit/s3	@aws-sdk/client-s3, @aws-sdk/lib-storage, @aws-sdk/s3-request-presigner
MinIO	storagekit/minio	minio
Azure Blob Storage	storagekit/azure	@azure/storage-blob
Oracle OCI Object Storage	storagekit/oracle	oci-objectstorage, oci-common

All provider SDKs are optional peer dependencies. Import only the entrypoint you use; when a driver is loaded without its SDK installed you get a clear StorageInvalidConfigError with the exact npm install command instead of a module crash.

Requires Node.js ≥ 20. Ships dual ESM + CJS with TypeScript declarations. Browser usage is not a goal: server-side credentials and filesystem access are first-class here.

Runtime support

Runtime	Status	Notes
Node.js ≥ 20	fully supported	primary target; CI matrix on 20 + 22
Bun ≥ 1.1	fully supported	smoke-tested on every build (CI job)
Deno ≥ 2.0	fully supported	via npm compatibility; smoke-tested on every build (CI job)

The runtime smoke (scripts/runtime-smoke.mjs) exercises the local driver, streams, listings, error normalization and the custom-driver registry on all three runtimes. Browser usage is not a goal: server-side credentials and filesystem access are first-class here.

Documentation

Rendered site: mohamedhabibwork.github.io/storagekit — auto-deployed from docs/ on every push to main.

Full per-driver guides live in docs/ (also shipped in the npm tarball):

Guide	Contents
Site: mohamedhabibwork.github.io/storagekit	rendered MkDocs + Material site (search, dark mode, copy-button)
docs/local.md	config, permissions, traversal protection, symlinks, native options
docs/s3.md	AWS SDK v3, multipart tuning, storage classes/KMS, presigned URLs, LocalStack testing
docs/minio.md	native MinIO client, metadata bags, copy preconditions, presigned URLs
docs/azure.md	auth routes, access tiers, SAS generation, versioning, Azurite testing
docs/oracle.md	OCI auth providers, native multipart, PARs, cross-region copy
docs/custom-drivers.md	full StorageDriver reference, registry semantics, contract testing, correctness checklist
docs/uploads.md	framework upload recipes: multer/Express/NestJS, Fastify, Hono, Next.js, formidable, web File

The design rule

Common operations share one API. Provider-specific behavior stays under the strongly-typed native key, based on the configured storage type.

{
  // ── normalized (same everywhere) ──
  contentType: 'image/jpeg',
  metadata: { userId: '100' },
  overwrite: false,
  multipart: { partSize: 10 * 1024 * 1024, concurrency: 4 },
  signal: controller.signal,

  // ── provider-specific (typed by the storage type) ──
  native: {
    // Storage<'s3'>: PutObjectCommandInput fields (StorageClass, ACL, SSEKMSKeyId, …)
    // Storage<'azure'>: { tier: 'Cool', conditions, tags, blobHTTPHeaders, … }
    // Storage<'minio'>: { metaData: { … } }
    // Storage<'oracle'>: { storageTier: 'Archive', ifMatch, opcMeta, … }
    // Storage<'local'>: { mode: 0o600 }
  },
}

native is merged last: it overrides the common equivalents when both are given.

Quick start (every provider)

import { createStorage } from '@mohamedhabibwork/storagekit';

// Local
const local = await createStorage({
  type: 'local',
  root: './storage/app',
  baseUrl: 'https://cdn.example.com', // optional, powers getUrl()
});

// AWS S3 (credentials resolve through the normal AWS chain when omitted)
const s3 = await createStorage({
  type: 's3',
  bucket: 'my-files',
  region: 'eu-west-1',
});

// MinIO
const minio = await createStorage({
  type: 'minio',
  bucket: 'uploads',
  endPoint: 'localhost',
  port: 9000,
  useSSL: false,
  accessKey: process.env.MINIO_ACCESS_KEY,
  secretKey: process.env.MINIO_SECRET_KEY,
});

// Azure Blob (connection string, shared key, TokenCredential, or inject clients)
const azure = await createStorage({
  type: 'azure',
  container: 'uploads',
  connectionString: process.env.AZURE_STORAGE_CONNECTION_STRING,
});

// Oracle — native auth providers, never access-key style
const oracle = await createStorage({
  type: 'oracle',
  namespaceName: 'mynamespace',
  bucketName: 'mybucket',
  region: 'eu-frankfurt-1',
  auth: { type: 'instance-principals' },
});

You can also inject pre-built native clients for dependency injection: client (S3 / MinIO / Oracle), serviceClient / containerClient (Azure), or authProvider (Oracle).

Core operations

// upload — Buffer, string, Uint8Array, ArrayBuffer, Blob or Node stream
await storage.upload('users/1/avatar.jpg', buffer, {
  contentType: 'image/jpeg',
});

// upload from a stream — never buffered into memory
await storage.upload('videos/movie.mp4', readableStream, {
  multipart: { enabled: true, partSize: 10 * 1024 * 1024, concurrency: 4 },
});

// download — stream-first
const download = await storage.download('documents/report.pdf');
download.stream.pipe(response);          // Node Readable
const text = await download.text();       // or .buffer() / .json()
download.contentType;                     // normalized metadata
download.etag; download.contentLength; download.metadata;

// existence & metadata (native HEAD calls, never full downloads)
await storage.exists('users/1/avatar.jpg');
const stat = await storage.stat('users/1/avatar.jpg'); // size, etag, contentType, …

// delete (idempotent — deleting a missing object is a no-op)
await storage.delete('users/1/avatar.jpg');
await storage.deleteMany(['a.jpg', 'b.jpg', 'c.jpg']); // per-path outcome report

// list — one level (default) or recursive, with opaque cursors
const page = await storage.list({ prefix: 'users/100/', limit: 100 });
page.files;        // StorageFile[] (path, size, etag, lastModified, …)
page.directories;  // ['users/100/sub/'] — trailing slash, one level
page.cursor;       // pass back via { cursor } to continue
page.hasMore;

// async iteration for very large buckets — pagination handled for you
for await (const file of storage.iterate('uploads/')) {
  console.log(file.path);
}

// copy (server-side) & move (copy + delete; rename on local)
await storage.copy('temp/image.jpg', 'images/image.jpg');
await storage.move('temp/file.pdf', 'documents/file.pdf');

Framework uploads (multer, Fastify, Hono, formidable, …)

storagekit plugs straight into the upload middleware of any framework — the file streams from the request into your storage, with safe UUID keys derived from the client filename. No adapter imports its framework, so multer, fastify, hono and formidable all stay optional peers.

// Express / NestJS / koa-multer — a real multer storage engine
import multer from 'multer';
import { createMulterStorage } from '@mohamedhabibwork/storagekit/adapters/express';

const upload = multer({ storage: createMulterStorage(storage, { directory: 'uploads' }) });
app.post('/upload', upload.single('avatar'), (req, res) => {
  res.json({ key: req.file!.key, etag: req.file!.etag, name: req.file!.originalname });
});

// Fastify (@fastify/multipart), Hono / Next.js / Bun / Deno (web File),
// formidable — see docs/uploads.md for each recipe.
import { saveFastifyFile } from '@mohamedhabibwork/storagekit/adapters/fastify';
import { saveWebFile } from '@mohamedhabibwork/storagekit/uploads';

saveUpload() is the universal intake: give it { body, originalName, mimeType } from any middleware and it resolves a traversal-safe key, derives contentType + metadata and streams the body to any driver.

URLs

// public URL — never performs network requests
await storage.getUrl('images/logo.png');

// signed URLs — read (default) / write / delete
await storage.getSignedUrl('documents/private.pdf', { expiresIn: 3600 });
await storage.getSignedUrl('upload-target.bin', {
  action: 'write',
  expiresIn: 900,
  native: { ResponseContentDisposition: 'attachment; filename="invoice.pdf"' }, // S3
});

• S3 / MinIO: presigned URLs with full native options.
• Azure: SAS tokens from shared-key credentials or connection strings. TokenCredential-based signing needs user delegation keys — use nativeRequest() with getUserDelegationKey.
• Local: unsupported — serve files through your app (throw StorageUnsupportedOperationError).
• Oracle: no presigned URLs. The storagekit/oracle entrypoint adds a provider-specific createPreauthenticatedRequest() because PARs have a fundamentally different lifecycle (persistent server-side resources).

Expiry is validated: between 1 second and 7 days (expiresIn).

Escaping the abstraction

// the real native client, typed per provider
const client = storage.native(); // S3Client | ContainerClient | Minio.Client | ObjectStorageClient | LocalNativeClient

// any SDK operation the package does not wrap
const info = await storage.nativeRequest((c) => c.someAdvancedSdkCall());

// Azure versioned downloads, Oracle PARs, S3 Select, … all stay reachable

Multiple disks

import { createStorageManager } from 'storagekit';

const disks = createStorageManager({
  default: 'uploads',
  disks: {
    uploads: { type: 's3', bucket: 'uploads' },
    backup: { type: 'azure', container: 'backup', accountUrl: 'https://acct.blob.core.windows.net' },
    temp: { type: 'local', root: './storage/temp' },
  },
});

await disks.disk('uploads').upload('a.txt', '…');   // Storage<'s3'>
await disks.disk('backup').upload('a.txt', '…');    // Storage<'azure'>
await disks.disk('temp').delete('a.txt');           // Storage<'local'>

Cross-storage copy

import { copyBetween } from 'storagekit';

await copyBetween(sourceStorage, 'docs/file.pdf', destinationStorage, 'archive/file.pdf', {
  concurrency: 4,
  onProgress: (bytes, total) => console.log(`${bytes}/${total ?? '?'}`),
});

Streams source → destination; the file is never fully buffered. For copies inside one provider prefer storage.copy() (server-side).

Errors

import {
  StorageError,
  StorageNotFoundError,
  StoragePermissionError,
  StorageConflictError,
  StorageInvalidConfigError,
  StorageNetworkError,
  StorageQuotaError,
  StorageUnsupportedOperationError,
  StorageInvalidPathError,
} from 'storagekit';

try {
  await storage.download(path);
} catch (error) {
  if (error instanceof StorageNotFoundError) {
    // handle missing object
  }
  error.provider;   // 's3' | 'local' | …
  error.operation;  // 'download'
  error.path;
  error.code;       // provider-native code when available
  error.cause;      // the original SDK error
}

Provider errors are normalized into the classes above while the original error stays on cause. Deleting missing objects is idempotent everywhere.

Prefixes

const storage = await createStorage({
  type: 's3',
  bucket: 'application',
  prefix: 'production/',
});

await storage.upload('users/avatar.jpg', file);
// stored as production/users/avatar.jpg

Prefix behavior is identical across providers — every operation (upload, download, list, copy, URLs, …) applies and strips it consistently. Local paths additionally can never escape the configured root (../../etc/passwd throws StorageInvalidPathError).

Hooks & observability

const storage = await createStorage(config, {
  hooks: {
    beforeUpload: (ctx) => log.debug('uploading', ctx.path),
    afterUpload: (ctx) => metrics.count('upload'),
    uploadError: (ctx) => alert(ctx.error),
    beforeDelete: (ctx) => audit(ctx.path),
  },
  onOperation: (event) => {
    event.provider; event.operation; event.duration; event.success;
  },
});

// subscribe later
const off = storage.on('operation', (event) => { … });
off(); // unsubscribe

Hooks receive sanitized contexts — credentials and signed URLs never appear.

Versioning & encryption

Provider semantics differ and are not hidden: download/stat/delete accept a normalized versionId (S3, Azure, MinIO, Oracle), while native controls (S3 ServerSideEncryption: 'aws:kms', Azure access tiers, OCI storageTier) remain available through native.

Custom drivers

Register your own storage type against the same unified API. Implement the StorageDriver interface, register it, and createStorage resolves it by its type string — native slots are typed unknown for custom types.

import {
  createStorage,
  registerStorageDriver,
  defineDriver,
  type StorageDriver,
} from '@mohamedhabibwork/storagekit';

const driver = defineDriver({
  type: 'memory',
  async upload(path, body, options) { /* … */ return { path, provider: 'memory' }; },
  async download(path) { /* → { stream, buffer(), text(), json() } */ },
  async delete(path) {},
  async deleteMany(paths, options) { /* → { deleted, failed } */ },
  async exists(path) { return false; },
  async stat(path) { /* → FileStat */ },
  async list(options) { /* → { files, directories, cursor, hasMore } */ },
  async copy(source, destination, options) {},
  async move(source, destination, options) {},
  async getUrl(path) { return ''; },
  async getSignedUrl(path, options) { throw new Error('unsupported'); },
  native() { return store; },
  nativeRequest(fn) { return fn(store); },
  capabilities() {
    return {
      signedUrls: false, multipartUpload: true, serverSideCopy: true,
      versioning: false, metadata: true, directories: false, bulkDelete: false,
    };
  },
});

registerStorageDriver('memory', (config) => driver);

const storage = await createStorage({ type: 'memory', /* custom config fields */ });

Registration is global to the process, cannot collide with builtin types, and an optional async ready() method on the driver is awaited by the factory. Validate your driver against the published contract suite (below).

Testing your own driver

The shared contract suite is published so custom drivers can prove they behave like the built-ins:

import { defineDriverContractTests } from '@mohamedhabibwork/storagekit/testing';

defineDriverContractTests({
  name: 'my-driver',
  createStorage: () => createMyStorage(config),
  capabilities: { signedUrls: true },
});

Development

npm install
npm test              # unit + contract + mocked-OCI tests
npm run test:types    # type-level assertions
npm run typecheck     # strict tsc
npm run build         # dual ESM/CJS + d.ts via tsup

Cloud integration tests are env-gated (see tests/integrations/*.ts): LocalStack for S3, minio/minio Docker for MinIO, Azurite for Azure, and opt-in live OCI (OCI_INTEGRATION_TESTS=true).

Security notes

• Credentials, connection strings, SAS tokens and signed URLs are never logged or embedded in normalized error messages.
• Local driver paths are validated against traversal and escape the root with StorageInvalidPathError.
• Signed URL lifetimes are bounded (1 s – 7 days).
• Zero runtime dependencies: every provider SDK is an optional peer dependency, so installing storagekit adds nothing to your supply chain beyond what you already chose to install.
• Releases are published from GitHub Actions with npm provenance — a verifiable, signed link between the published tarball and this repository's build.
• Continuous supply-chain analysis by Socket: the two informational alerts on the package itself — Filesystem access (the local driver is built on fs/fs/promises) and URL strings (hardcoded provider endpoint domains such as amazonaws.com, core.windows.net, oraclecloud.com) — are inherent to what a storage library does, not vulnerabilities. Alerts you may see under the "dependencies" tab belong to the dev-time installs of the optional peer SDKs (mostly Oracle's SDK tree) and are never installed by consumers of this package.

License

MIT