multerator

Overview

Multerator (short for multipart-iterator) is a multipart/form-data parser for Node.js.

Compatible for Node.js versions >= 10.21.0.

This is an initial README and more documentation will be eventually added.

Installation

With npm:

npm install multerator

With yarn:

yarn add multerator

Synopsis

const { multerator } = require('multerator');

(async () => {
   // Obtain a multipart data stream:
  const stream = getSomeMultipartStream();
  const boundary = '--------------------------120789128139917295588288';

  // Feed it to multerator:
  const streamParts = multerator({ input: stream, boundary });

  for await (const part of streamParts) {
    if (part.type === 'text') {
      console.log(
        `Got text field "${part.name}" with value "${part.data}"`
      );
    } else {
      console.log(
        `Got file field "${part.name}" of filename "${part.filename}" with content type "${part.contentType}" and incoming data chunks:`
      );
      for await (const chunk of part.data) {
        console.log(`Received ${chunk.length} bytes`);
      }
    }
  }
})();

API

Input parameters

Name	Type	Description
options	object (required)
options.input	Readable | AsyncIterable<Buffer> (required)	A Readable stream or any async iterable of Buffer objects.
options.boundary	string (required)	The boundary token by which to separate parts across the contents of given options.input.
options.maxFileSize	number	Default: none. Optional size limit (in bytes) for individual file part bodies. The moment this limit is reached, multerator will immediately cut the input data stream and yield an error of type ERR_BODY_REACHED_SIZE_LIMIT.
options.maxFieldSize	number	Default: none. Optional size limit (in bytes) for individual field part bodies. The moment this limit is reached, multerator will immediately cut the input data stream and yield an error of type ERR_BODY_REACHED_SIZE_LIMIT. That's a recommended general safety measure as field part bodies are collected as complete strings in memory which might be unsafe in the case of dealing with an "unreasonable" data source.

Usage examples

General usage:

const fs = require('fs');
const { PassThrough } = require('stream');
const FormData = require('form-data');
const { multerator } = require('multerator');

(async () => {
  // Obtain a multipart data stream with help from form-data package:
  const form = new FormData();
  form.append('my_text_field', 'my text value');
  form.append('my_file_field', fs.createReadStream(`${__dirname}/image.jpg`));
  const input = form.pipe(new PassThrough()); // Converting the form data instance into a normalized Node.js stream, which is async-iteration-friendly as required for multerator's input
  const boundary = form.getBoundary();

  // Feed it to multerator:
  try {
    for await (const part of multerator({ input, boundary })) {
      if (part.type === 'text') {
        console.log(
          `Got text field "${part.name}" with value "${part.data}"`
        );
      } else {
        console.log(
          `Got file field "${part.name}" of filename "${part.filename}" with content type "${part.contentType}" and incoming data chunks:`
        );
        for await (const chunk of part.data) {
          console.log(`Received ${chunk.length} bytes`);
        }
      }
    }
  } catch (err) {
    console.log('Multipart parsing failed:', err);
  }
})();

Very banal file upload server with Express:

const { createWriteStream } = require('fs');
const { pipeline } = require('stream');
const { promisify } = require('util');
const express = require('express');
const { multerator } = require('multerator');

const pipelinePromisified = promisify(pipeline);

const expressApp = express();

expressApp.post('/upload', async (req, res) => {
  const contentType = req.headers['content-type'];

  try {
    if (!contentType.startsWith('multipart/form-data')) {
      throw new Error(
        '😢 Only requests of type multipart/form-data are allowed'
      );
    }

    const boundary = contentType.split('boundary=')[1];

    const parts = multerator({ input: req, boundary });

    for await (const part of parts) {
      if (part.type === 'file') {
        console.log(
          `Incoming upload: field name: ${part.name}, filename: ${part.filename}, content type: ${part.contentType}`
        );
        await pipelinePromisified(
          part.data,
          createWriteStream(`${__dirname}/uploads/${part.filename}`)
        );
      }
    }

    res.status(200).send({ success: true });
  } catch (err) {
    res.status(500).send({ success: false, error: err.message });
  }
});

expressApp.listen(8080, () => console.log('Server listening on 8080'));

...callable by e.g:

curl \
  -F my_file_field=@image.jpg \
  http://127.0.0.1:8080/upload

Changelog

0.11.0 (2022-10-23)

⚠ BREAKING CHANGES

• normalize main library export (#59)

Features

• add parseTextFields optional boolean parameter (true on default) to make it possible to disable the text field parsing behavior (#50) (a393581)
• add dual CommonJS and ESM builds (#58) (34b107f)
• improve the main Multerator iterable's return type to be externally more type-forgiving (#57) (0fd78b8)
• normalize main library export (#59) (3b0c461)

Bug Fixes

• ending a part body's iterable should also close the source (#55) (74b6961)
• ending the multerator iterable not ending the source input (#48) (69d27aa)
• failed detecting start boundaries that aren't preceded by a CRLF (#45) (f3a7b97)