@tweedegolf/sab-adapter-amazon-s3

Amazon S3 Adapter

This adapter is a peer dependency of the storage abstraction package. It provides an abstraction layer over the API of the Amazon S3 cloud storage service and S3 compatible services like Cubbit, Cloudflare R2 and Backblaze B2 S3.

If you are new to the Storage Abstraction library you may want to read this first.

import { Storage, StorageType } from "@tweedegolf/storage-abstraction";

const configuration = {
  type: StorageType.S3,
};

const storage = new Storage(configuration);

const result = await a.listBuckets();

console.log(result);

When you create a Storage instance it checks the mandatory type key in the configuration object and then loads the appropriate adapter module automatically from your node_modules folder using require. For more information please read this.

Configuration

The configuration object that you pass to the Storage constructor is forwarded to the constructor of the adapter.

The Storage constructor is only interested in the type key of the configuration object, all other keys are necessary for configuring the adapter.

The Storage constructor expects the configuration to be of type StorageAdapterConfig.

The adapter expects the configuration to be of type AdapterConfig or a type that extends this AdapterConfig.

export interface AdapterConfig {
  bucketName?: string;
  [id: string]: any; // any mandatory or optional key
}

export interface StorageAdapterConfig extends AdapterConfig {
  type: string;
}

The type of the configuration object for this adapter:

export interface AdapterConfigS3 extends AdapterConfig {
  region?: string;
  endpoint?: string;
  credentials?: {
    accessKeyId?: string;
    secretAccessKey?: string;
  };
  accessKeyId?: string;
  secretAccessKey?: string;
}

Example with configuration object:

// Cubbit S3 compatible
const s = new Storage({
  type: StorageType.S3,
  accessKeyId: 'your-key-id'
  secretAccessKey: 'your-secret'
  endpoint: "https://s3.cubbit.eu/",
  region: "auto",
});

Example with configuration url:

// Cubbit S3 compatible
const s = new Storage(
  "s3://accessKeyId=your-key-id&secretAccessKey=your-access-key&endpoint=https://s3.cubbit.eu/&region=auto"
);

Amazon S3

If you use this adapter to interact with the original Amazon S3 service it is possible to skip the passing in of the accessKeyId, secretAccessKey and region; the AWS SDK will automatically read it from a chain of providers, e.g. from environment variables or the ECS task role, so this will work:

// only for Amazon S3
const s = new Storage({ type: StorageType.S3 });
// with a config url:
const s = new Storage("s3://");
// and even:
const s = new Storage("s3");

The environment variables that you need to set for this are:

AWS_ACCESS_KEY_ID="your access key"
AWS_SECRET_ACCESS_KEY="your secret"
AWS_REGION="eu-west-1"

Note that this does not work for S3 compatible services because the AWS SDK doesn't read the endpoint from environment variables.

Also, if you pass a value for endpoint in the config, for some reason AWS SDK does read the environment variables AWS_REGION AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY anymore.

So for S3 compatible services setting a value for endpoint, accessKeyId and secretAccessKey in the config is mandatory.

For S3 compatible services region is mandatory as well but you don't have to pass this in the config because AWS SDK always reads the AWS_REGION environment variable if no value is provided in the config. Note that the names of the regions may differ from service to service, see below.

S3 Compatible Storage

Cloudflare R2, Backblaze B2 and Cubbit are S3 compatible services. You can use the AdapterAmazonS3 but you have to add a value for endpoint in the config.

Cloudflare R2

const s = new Storage({
  type: StorageType.S3,
  region: 'auto'
  endpoint: process.env.R2_ENDPOINT,
  accessKeyId: process.env.R2_ACCESS_KEY,
  secretAccessKey: process.env.R2_SECRET_KEY,
});

The endpoint is https://<ACCOUNT_ID>.<JURISDICTION>.r2.cloudflarestorage.com.

Jurisdiction is optional, e.g. eu.

It is mandatory to set a value for region, use one of these values:

• auto
• wnam
• enam
• weur
• eeur
• apac

You can also set the region using the AWS_REGION environment variable.

Backblaze S3

const s = new Storage({
  type: StorageType.S3,
  region: "eu-central-003",
  endpoint: process.env.B2_ENDPOINT,
  accessKeyId: process.env.B2_APPLICATION_KEY_ID,
  secretAccessKey: process.env.B2_APPLICATION_KEY,
});

The endpoint is https://s3.<REGION>.backblazeb2.com. Although the region is part of the endpoint AWS SDK still expects you to set a value for region in the configuration or in the AWS_REGION environment variable. As just stated, you can simply retrieve your region from the endpoint.

Backblaze also has a native API. You can use this adapter if you want to use the native API.

API

For a complete description of the Adapter API see this part documentation of the Storage Abstraction package readme.

Standalone

You can also use the adapter standalone, without the need to create a Storage instance:

import { AdapterAmazonS3 } from "@tweedegolf/sab-adapter-amazon-s3";

const a = new AdapterAmazonS3();
const r = await a.listBuckets();
console.log(r);