@constructive-io/s3-utils

s3-utils

Unified S3 utilities for the Constructive ecosystem — client factory, file operations, presigned URLs, and bucket management. Built on AWS SDK v3 with support for multiple S3-compatible providers.

Features

• Multi-provider support — AWS S3, MinIO, Cloudflare R2, Google Cloud Storage, DigitalOcean Spaces
• Presigned URLs — generate secure PUT and GET URLs with configurable expiry
• File operations — streaming upload, download, existence checks, and metadata retrieval
• Bucket management — create buckets with provider-appropriate policies and CORS
• AWS SDK v3 — modern, modular SDK with tree-shaking support

Installation

npm install @constructive-io/s3-utils

Quick Start

import {
  createS3Client,
  presignPutUrl,
  presignGetUrl,
  upload,
  download,
} from '@constructive-io/s3-utils';

// 1. Create a client
const client = createS3Client({
  provider: 'minio',
  region: 'us-east-1',
  endpoint: 'http://minio:9000',
  accessKeyId: 'minioadmin',
  secretAccessKey: 'minioadmin',
});

// 2. Upload a file via stream
import { createReadStream } from 'fs';

const result = await upload({
  client,
  bucket: 'my-bucket',
  key: 'docs/report.pdf',
  contentType: 'application/pdf',
  readStream: createReadStream('/path/to/report.pdf'),
});

// 3. Generate a presigned download URL
const url = await presignGetUrl(client, {
  bucket: 'my-bucket',
  key: 'docs/report.pdf',
});

API

createS3Client(config)

Creates an S3Client with provider-specific defaults.

import { createS3Client } from '@constructive-io/s3-utils';

const client = createS3Client({
  provider: 'r2',
  region: 'auto',
  endpoint: 'https://account.r2.cloudflarestorage.com',
  accessKeyId: 'KEY',
  secretAccessKey: 'SECRET',
});

StorageConnectionConfig

Field	Type	Required	Description
provider	's3' | 'minio' | 'r2' | 'gcs' | 'spaces'	yes	Storage provider
region	string	yes	S3 region (e.g. "us-east-1")
endpoint	string	non-AWS	Endpoint URL (required for all providers except s3)
accessKeyId	string	yes	Access key ID
secretAccessKey	string	yes	Secret access key
forcePathStyle	boolean	no	Override path-style URL behavior

Path-style URLs are enabled automatically for minio, r2, and gcs. Virtual-hosted style is used for s3 and spaces.

Throws S3ConfigError on invalid configuration.

presignPutUrl(client, options)

Generates a presigned PUT URL for uploading. The URL is locked to the specified key, content type, and content length.

const url = await presignPutUrl(client, {
  bucket: 'my-bucket',
  key: 'uploads/photo.jpg',
  contentType: 'image/jpeg',
  contentLength: 102400,
  expiresIn: 900, // default: 900 (15 min)
});

presignGetUrl(client, options)

Generates a presigned GET URL for downloading. Supports an optional filename for Content-Disposition.

const url = await presignGetUrl(client, {
  bucket: 'my-bucket',
  key: 'uploads/photo.jpg',
  expiresIn: 3600, // default: 3600 (1 hour)
  filename: 'photo.jpg', // optional, triggers download prompt
});

headObject(client, bucket, key)

Checks if an object exists and returns its metadata. Returns null if the object is not found.

const meta = await headObject(client, 'my-bucket', 'uploads/photo.jpg');
// { contentType: 'image/jpeg', contentLength: 102400 } or null

fileExists({ client, bucket, key })

Returns true if the object exists, false otherwise.

const exists = await fileExists({ client, bucket: 'my-bucket', key: 'doc.pdf' });

upload({ client, bucket, key, readStream, contentType })

Streams a file to S3 using multipart upload.

import { createReadStream } from 'fs';

const result = await upload({
  client,
  bucket: 'my-bucket',
  key: 'uploads/video.mp4',
  contentType: 'video/mp4',
  readStream: createReadStream('/path/to/video.mp4'),
});
// { Location, ETag, Bucket, Key }

download({ client, bucket, key, writeStream })

Downloads an object from S3 and pipes it to a writable stream.

import { createWriteStream } from 'fs';

await download({
  client,
  bucket: 'my-bucket',
  key: 'uploads/video.mp4',
  writeStream: createWriteStream('/tmp/video.mp4'),
});

uploadThrough({ client, bucket, key, contentType })

Returns a PassThrough stream that uploads to S3 as data is written. Emits an 'upload' event with the result when complete.

const pass = uploadThrough({
  client,
  bucket: 'my-bucket',
  key: 'stream-data.csv',
  contentType: 'text/csv',
});

pass.on('upload', (result) => {
  console.log('Upload complete:', result.Location);
});

someReadable.pipe(pass);

createS3Bucket(client, bucket, options)

Creates a bucket with provider-appropriate policies and CORS configuration.

• MinIO: read-only public access policy (list + get)
• S3/GCS: full-access policy with CORS rules for browser uploads

const { success } = await createS3Bucket(client, 'my-bucket', {
  provider: 'minio',
});

Returns { success: true } if the bucket was created or already exists.

Related Packages

Package	Description
@constructive-io/s3-streamer	Streaming uploads with automatic content-type detection and metadata extraction
@constructive-io/etag-stream	ETag generation from streams
@constructive-io/content-type-stream	Magic-byte content-type detection
@constructive-io/uuid-hash	Deterministic UUID generation from file content

License

MIT

Education and Tutorials

• 🚀 Quickstart: Getting Up and Running Get started with modular databases in minutes. Install prerequisites and deploy your first module.

• 📦 Modular PostgreSQL Development with Database Packages Learn to organize PostgreSQL projects with pgpm workspaces and reusable database modules.

• ✏️ Authoring Database Changes Master the workflow for adding, organizing, and managing database changes with pgpm.

• 🧪 End-to-End PostgreSQL Testing with TypeScript Master end-to-end PostgreSQL testing with ephemeral databases, RLS testing, and CI/CD automation.

• ⚡ Supabase Testing Use TypeScript-first tools to test Supabase projects with realistic RLS, policies, and auth contexts.

• 💧 Drizzle ORM Testing Run full-stack tests with Drizzle ORM, including database setup, teardown, and RLS enforcement.

• 🔧 Troubleshooting Common issues and solutions for pgpm, PostgreSQL, and testing.

Related Constructive Tooling

📦 Package Management

• pgpm: 🖥️ PostgreSQL Package Manager for modular Postgres development. Works with database workspaces, scaffolding, migrations, seeding, and installing database packages.

🧪 Testing

• pgsql-test: 📊 Isolated testing environments with per-test transaction rollbacks—ideal for integration tests, complex migrations, and RLS simulation.
• pgsql-seed: 🌱 PostgreSQL seeding utilities for CSV, JSON, SQL data loading, and pgpm deployment.
• supabase-test: 🧪 Supabase-native test harness preconfigured for the local Supabase stack—per-test rollbacks, JWT/role context helpers, and CI/GitHub Actions ready.
• graphile-test: 🔐 Authentication mocking for Graphile-focused test helpers and emulating row-level security contexts.
• pg-query-context: 🔒 Session context injection to add session-local context (e.g., SET LOCAL) into queries—ideal for setting role, jwt.claims, and other session settings.

🧠 Parsing & AST

• pgsql-parser: 🔄 SQL conversion engine that interprets and converts PostgreSQL syntax.
• libpg-query-node: 🌉 Node.js bindings for libpg_query, converting SQL into parse trees.
• pg-proto-parser: 📦 Protobuf parser for parsing PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums.
• @pgsql/enums: 🏷️ TypeScript enums for PostgreSQL AST for safe and ergonomic parsing logic.
• @pgsql/types: 📝 Type definitions for PostgreSQL AST nodes in TypeScript.
• @pgsql/utils: 🛠️ AST utilities for constructing and transforming PostgreSQL syntax trees.

Credits

🛠 Built by the Constructive team — creators of modular Postgres tooling for secure, composable backends. If you like our work, contribute on GitHub.

Disclaimer

AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND.

No developer or entity involved in creating this software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the code, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value.