@appgeist/storage

@appgeist/storage

An opinionated Express-based storage server featuring assets uploading and on-demand image-resizing. Like a self-hosted Cloudinary.

Uses ImageMagick, multer and UUID.

Requires ImageMagick with WebP support installed. Most linux distributions provide it by default and brew install imagemagick is everything you need to run on macOS.

Why

Because:

• Sometimes you want your assets to be stored on your own server;
• Most browsers can display WebP images and you should definitely use the new format whenever possible instead of jpg, png or gif;
• You need your pictures in various sizes and/or formats for different screen resolutions, sizes and device capabilities, but you don't always know all the possible combinations a priori.

Usage

With default options:

const express = require("express");
const storage = require("@appgeist/storage");

const app = express();
app.use("/assets", storage()); // '/assets' is the storage server mountpoint

app.listen(3000, err => {
  if (err) throw err;
  // eslint-disable-next-line no-console
  console.log("Storage server running...");
});

With custom config options:

const express = require("express");
const storage = require("@appgeist/storage");

const app = express();
app.use(
  "/assets",
  storage({
    // '/assets' is the storage server mountpoint
    logPattern: PRODUCTION ? ":method :url :status - :response-time ms" : "dev",
    rootDir: "./files",
    tmpDir: "./temp-uploads",
    maxFileSize: 1024 * 1024 * 50, // 50 megabytes
    maxPicturePixels: 3840 * 2160, // 4K
    tokenValidationHandler: async token => {
      // a method to validate upload requests
      // ...
      const isValid = await myValidationApiMethod(token);
      return isValid;
    }
  })
);

app.listen(3000, err => {
  if (err) throw err;
  // eslint-disable-next-line no-console
  console.log("Storage server running...");
});

See storage-example-simple and storage-example-with-auth for more.

Default config options

See index.js for more info on the default config options. JSDoc comments are also provided for IDE support.

Uploading files

Request payload

Files can be uploaded by POSTing to the mountpoint or a subfolder (i.e. POST /assets or POST /assets/a/subfolder/to/store/the/uploaded/files) with the following body payload:

{
  file: fileData;
}

...where file represents the file multipart/form-data (provided, for instance by an <input type="file" /> tag, see multer docs for more info).

Instead of simply uploading a file, the server can be instructed to dowload it from an accesible remote location by POSTing a URL instead of file data:

{
  url: "http://example.com/catz.jpg";
}

The uploaded file will end up in ${rootDir}/assets or ${rootDir}/assets/a/subfolder/to/store/the/uploaded/files respectively.

A UUID/v4-based name will be generated for the uploaded file.

If the uploaded file is an image (jpg/webp/png/gif), it will be converted to webp using ImageMagick library and resized to maxPicturePixels, otherwise it will simply be stored in the original format.

Examples:

• uploading catz.jpg to /assets will generate a file like /assets/9752d427-e6e2-4868-8abf-720db82421c2.webp;
• uploading doc.pdf to /assets/docs will generate a file like /assets/docs/9752d427-e6e2-4868-8abf-720db82421c2.pdf;

Token-based authorization

Bearer token based authorization can be enabled by setting a tokenValidationHandler in the initialization options to a function. The tokenValidationHandler method will receive the upload request bearer token (specified as Authentication: Bearer <token> header) as its only parameter and must return a boolean or a Promise resolving to a boolean value.

Server response

When succesfully uploading catz.jpg to /assets/pics/animals, the server will respond with the following JSON:

{
  "path": "/pics/animals",
  "uuid": "9752d427-e6e2-4868-8abf-720db82421c2",
  "isPicture": true,
  "originalName": "catz.jpg",
  "aspectRatio": 1.77778
}

...where:

• path is where the the image was uploaded;
• uuid is a RFC-4122 unique identifier generated by UUID/v4;
• isPicture is true (and false for non-picture uploads);
• originalName is the original file name (or URL);
• aspectRatio is the picture aspect ratio (determined by ImageMagick identify utility; this property is only present for picture file uploads).

Serving files

Stored picture files can be served in multiple formats, resized to fit or centered/cropped to a specified size.

Assuming a 9752d427-e6e2-4868-8abf-720db82421c2 uuid was generated by a previous picture upload, a file will be generated and served to GET requests like this:

• /9752d427-e6e2-4868-8abf-720db82421c2.webp will serve the original picture;
• /9752d427-e6e2-4868-8abf-720db82421c2-800x600.webp will serve the picture resized and centered/cropped to a width of 800px and a height of 600px;
• /9752d427-e6e2-4868-8abf-720db82421c2-800x600-fit.webp will serve the picture resized to fit within a maximum width of 800px and a maximum height of 600px;
• /9752d427-e6e2-4868-8abf-720db82421c2-50x50-lq.webp will serve a low-quality picture resized and centered/cropped to 50x50 pixels (useful for LQIPs).

Cropping and resizing only work for pictures. Other types of assets will only be served in the original format.

Caution

Files generated for different sizes and formats are never deleted. This can quickly eat up your server space. Make sure to implement a mechanism to delete old/unnecessary files!!!