formidable
Advanced tools
Package description
The formidable npm package is a Node.js module for parsing form data, especially file uploads. It can handle multipart/form-data, which is used for uploading files through forms.
File Upload
This code creates an HTTP server that listens for POST requests on the '/upload' path. It uses formidable to parse the incoming form data and handle file uploads.
const formidable = require('formidable');
const http = require('http');
http.createServer((req, res) => {
if (req.url === '/upload' && req.method.toLowerCase() === 'post') {
const form = new formidable.IncomingForm();
form.parse(req, (err, fields, files) => {
if (err) {
res.writeHead(500, { 'content-type': 'text/plain' });
res.end('Error parsing the files');
return;
}
res.writeHead(200, { 'content-type': 'text/plain' });
res.write('Files uploaded successfully:\n');
res.end(JSON.stringify(files, null, 2));
});
}
}).listen(8080);Form Field Parsing
This code snippet demonstrates how to use formidable to parse regular form fields (text inputs, selects, etc.) in addition to file uploads.
const formidable = require('formidable');
const http = require('http');
http.createServer((req, res) => {
if (req.url === '/submit' && req.method.toLowerCase() === 'post') {
const form = new formidable.IncomingForm();
form.parse(req, (err, fields, files) => {
if (err) {
res.writeHead(500, { 'content-type': 'text/plain' });
res.end('Error parsing the form fields');
return;
}
res.writeHead(200, { 'content-type': 'text/plain' });
res.write('Form fields submitted:\n');
res.end(JSON.stringify(fields, null, 2));
});
}
}).listen(8080);File Upload Progress
This example shows how to track the progress of a file upload using formidable's 'progress' event, which provides the bytes received and the total bytes expected.
const formidable = require('formidable');
const http = require('http');
http.createServer((req, res) => {
if (req.url === '/upload' && req.method.toLowerCase() === 'post') {
const form = new formidable.IncomingForm();
form.on('progress', (bytesReceived, bytesExpected) => {
console.log(`Progress: ${bytesReceived}/${bytesExpected}`);
});
form.parse(req, (err, fields, files) => {
// Handle file upload and response
});
}
}).listen(8080);Multer is another popular Node.js middleware for handling multipart/form-data, which is primarily used for uploading files. It is built on top of busboy for maximum efficiency. Unlike formidable, multer is specifically designed for use with Express applications and includes more options for file storage and manipulation.
Busboy is a low-level Node.js module for parsing multipart/form-data request bodies. Formidable is actually built on top of busboy. Busboy is faster and more efficient but requires more setup and manual handling compared to formidable, which provides a higher-level API.
Multiparty is a Node.js module for parsing multipart/form-data requests. It is similar to formidable in terms of functionality but has a different API and is known for being more memory efficient, as it streams files to disk instead of buffering them in memory.
Readme
A Node.js module for parsing form data, especially file uploads.
This module was initially developed by @felixge for Transloadit, a service focused on uploading and encoding images and videos. It has been battle-tested against hundreds of GBs of file uploads from a large variety of clients and is considered production-ready and is used in production for years.
Currently, we are few maintainers trying to deal with it. :) More contributors are always welcome! :heart: Jump on issue #412 if you are interested.
Note: Master is a "canary" branch - try it with npm i formidable@canary.
Do not expect (for now) things from it to be inside thelatest"dist-tag" in the
Npm. Theformidable@latestis thev1.2.1 version and probably it will be the
lastv1 release!
Note: v2 is coming soon!
You can try the
Plugins API
(#545), which is
available through formidable@dev.
npm install formidable
# or the canary version
npm install formidable@canary
or with Yarn v1/v2
yarn add formidable
# or the canary version
yarn add formidable@canary
This is a low-level package, and if you're using a high-level framework it may already be included.
However, Express v4 does not include any multipart handling, nor does body-parser.
For koa there is koa-better-body which
can handle ANY type of body / form-data - JSON, urlencoded, multpart and so on.
A new major release is coming there too.
Parse an incoming file upload.
const http = require('http');
const util = require('util');
const formidable = require('formidable');
http
.createServer((req, res) => {
if (req.url === '/upload' && req.method.toLowerCase() === 'post') {
// parse a file upload
const form = formidable();
form.parse(req, (err, fields, files) => {
res.writeHead(200, { 'content-type': 'text/plain' });
res.write('received upload:\n\n');
res.end(util.inspect({ fields: fields, files: files }));
});
return;
}
// show a file upload form
res.writeHead(200, { 'content-type': 'text/html' });
res.end(`
<form action="/upload" enctype="multipart/form-data" method="post">
<input type="text" name="title" /><br/>
<input type="file" name="upload" multiple="multiple" /><br/>
<input type="submit" value="Upload" />
</form>
`);
})
.listen(8080, () => {
console.log('Server listening on http://localhost:8080/ ...');
});
The benchmark is quite old, from the old codebase. But maybe quite true though. Previously the numbers was around ~500 mb/sec. Currently with moving to the new Node.js Streams API it's faster. You can clearly see the differences between the Node versions.
Note: a lot better benchmarking could and should be done in future.
Benchmarked on 8GB RAM, Xeon X3440 (2.53 GHz, 4 cores, 8 threads)
~/github/node-formidable master
❯ nve --parallel 8 10 12 13 node benchmark/bench-multipart-parser.js
⬢ Node 8
1261.08 mb/sec
⬢ Node 10
1113.04 mb/sec
⬢ Node 12
2107.00 mb/sec
⬢ Node 13
2566.42 mb/sec
All shown are equivalent.
Please pass options to the function/constructor, not by passing
assigning them to the instance form
const formidable = require('formidable');
const form = formidable(options);
// or
const { formidable } = require('formidable');
const form = formidable(options);
// or
const { IncomingForm } = require('formidable');
const form = new IncomingForm(options);
// or
const { Formidable } = require('formidable');
const form = new Formidable(options);
See it's defaults in src/Formidable.js (the
DEFAULT_OPTIONS constant).
options.encoding {string} - default 'utf-8'; sets encoding for
incoming form fields,options.uploadDir {string} - default os.tmpdir(); the directory for
placing file uploads in. You can move them later by using fs.rename()options.keepExtensions {boolean} - default false; to include the
extensions of the original files or notoptions.maxFileSize {number} - default 200 * 1024 * 1024 (200mb);
limit the size of uploaded file.options.maxFields {number} - default 1000; limit the number of fields
that the Querystring parser will decode, set 0 for unlimitedoptions.maxFieldsSize {number} - default 20 * 1024 * 1024 (20mb);
limit the amount of memory all fields together (except files) can allocate in
bytes.options.hash {boolean} - default false; include checksums calculated
for incoming files, set this to some hash algorithm, see
crypto.createHash
for available algorithmsoptions.multiples {boolean} - default false; when you call the
.parse method, the files argument (of the callback) will contain arrays of
files for inputs which submit multiple files using the HTML5 multiple
attribute. Also, the fields argument will contain arrays of values for
fields that have names ending with '[]'.Note: If this value is exceeded, an 'error' event is emitted.
// The amount of bytes received for this form so far.
form.bytesReceived;
// The expected number of bytes in this form.
form.bytesExpected;
Parses an incoming Node.js request containing form data. If callback is
provided, all fields and files are collected and passed to the callback.
const formidable = require('formidable');
const form = formidable({ multiples: true, uploadDir: __dirname });
form.parse(req, (err, fields, files) => {
console.log('fields:', fields);
console.log('files:', files);
});
You may overwrite this method if you are interested in directly accessing the
multipart stream. Doing so will disable any 'field' / 'file' events
processing which would occur otherwise, making you fully responsible for
handling the processing.
In the example below, we listen on couple of events and direct them to the
data listener, so you can do whatever you choose there, based on whether its
before the file been emitted, the header value, the header name, on field, on
file and etc.
Or the other way could be to just override the form.onPart as it's shown a bit
later.
form.once('error', console.error);
form.on('fileBegin', (filename, file) => {
form.emit('data', { name: 'fileBegin', filename, value: file });
});
form.on('file', (filename, file) => {
form.emit('data', { name: 'file', key: filename, value: file });
});
form.on('field', (fieldName, fieldValue) => {
form.emit('data', { name: 'field', key: fieldName, value: fieldValue });
});
form.once('end', () => {
console.log('Done!');
});
// If you want to customize whatever you want...
form.on('data', ({ name, key, value, buffer, start, end, ...more }) => {
if (name === 'partBegin') {
}
if (name === 'partData') {
}
if (name === 'headerField') {
}
if (name === 'headerValue') {
}
if (name === 'headerEnd') {
}
if (name === 'headersEnd') {
}
if (name === 'field') {
console.log('field name:', key);
console.log('field value:', value);
}
if (name === 'file') {
console.log('file:', key, value);
}
if (name === 'fileBegin') {
console.log('fileBegin:', key, value);
}
});
A method that allows you to extend the Formidable library. By default we include 4 plugins, which esentially are adapters to plug the different built-in parsers.
The plugins added by this method are always enabled.
See src/plugins/ for more detailed look on default plugins.
The plugin param has such signature:
function(formidable: Formidable, options: Options): void;
The architecture is simple. The plugin is a function that is passed with the
Formidable instance (the form across the README examples) and the options.
Note: the plugin function's this context is also the same instance.
const formidable = require('formidable');
const form = formidable({ keepExtensions: true });
form.use((self, options) => {
// self === this === form
console.log('woohoo, custom plugin');
// do your stuff; check `src/plugins` for inspiration
});
form.parse(req, (error, fields, files) => {
console.log('done!');
});
Important to note, is that inside plugin this.options, self.options and
options MAY or MAY NOT be the same. General best practice is to always use the
this, so you can later test your plugin independently and more easily.
If you want to disable some parsing capabilities of Formidable, you can disable
the plugin which corresponds to the parser. For example, if you want to disable
multipart parsing (so the src/parsers/Multipart.js
which is used in src/plugins/multipart.js), then
you can remove it from the options.enabledPlugins, like so
const { Formidable } = require('formidable');
const form = new Formidable({
hash: 'sha1',
enabledPlugins: ['octetstream', 'querystring', 'json'],
});
Be aware that the order MAY be important too. The names corresponds 1:1 to files in src/plugins/ folder.
Pull requests for new built-in plugins MAY be accepted - for example, more
advanced querystring parser. Add your plugin as a new file in src/plugins/
folder (lowercased) and follow how the other plugins are made.
If you want to use Formidable to only handle certain parts for you, you can do something similar. Or see #387 for inspiration, you can for example validate the mime-type.
const form = formidable();
form.onPart = (part) => {
part.on('data', (buffer) {
// do whatever you want here
});
};
For example, force Formidable to be used only on non-file "parts" (i.e., html fields)
const form = formidable();
form.onPart = function(part) {
// let formidable handle only non-file parts
if (part.filename === '' || !part.mime) {
// used internally, please do not override!
form.handlePart(part);
}
};
export interface File {
// The size of the uploaded file in bytes.
// If the file is still being uploaded (see `'fileBegin'` event),
// this property says how many bytes of the file have been written to disk yet.
file.size: number;
// The path this file is being written to. You can modify this in the `'fileBegin'` event in
// case you are unhappy with the way formidable generates a temporary path for your files.
file.path: string;
// The name this file had according to the uploading client.
file.name: string | null;
// The mime type of this file, according to the uploading client.
file.type: string | null;
// A Date object (or `null`) containing the time this file was last written to.
// Mostly here for compatibility with the [W3C File API Draft](http://dev.w3.org/2006/webapi/FileAPI/).
file.lastModifiedDate: Date | null;
// If `options.hash` calculation was set, you can read the hex digest out of this var.
file.hash: string | 'sha1' | 'md5' | 'sha256' | null;
}
This method returns a JSON-representation of the file, allowing you to
JSON.stringify() the file which is useful for logging and responding to
requests.
'progress'Emitted after each incoming chunk of data that has been parsed. Can be used to roll your own progress bar.
form.on('progress', (bytesReceived, bytesExpected) => {});
'field'Emitted whenever a field / value pair has been received.
form.on('field', (name, value) => {});
'fileBegin'Emitted whenever a new file is detected in the upload stream. Use this event if you want to stream the file to somewhere else while buffering the upload on the file system.
form.on('fileBegin', (name, file) => {});
'file'Emitted whenever a field / file pair has been received. file is an instance of
File.
form.on('file', (name, file) => {});
'error'Emitted when there is an error processing the incoming form. A request that
experiences an error is automatically paused, you will have to manually call
request.resume() if you want the request to continue firing 'data' events.
form.on('error', (err) => {});
'aborted'Emitted when the request was aborted by the user. Right now this can be due to a
'timeout' or 'close' event on the socket. After this event is emitted, an
error event will follow. In the future there will be a separate 'timeout'
event (needs a change in the node core).
form.on('aborted', () => {});
'end'Emitted when the entire request has been received, and all contained files have finished flushing to disk. This is a great place for you to send your response.
form.on('end', () => {});
multipart_parser.js.If the documentation is unclear or has a typo, please click on the page's Edit
button (pencil icon) and suggest a correction. If you would like to help us fix
a bug or add a new feature, please check our
Contributing Guide. Pull requests are welcome!
Thanks goes to these wonderful people (emoji key):
Formidable is licensed under the MIT License.
FAQs
A node.js module for parsing form data, especially file uploads.
We found that formidable demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 5 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
CISA launched a new project called Vulnrichment to enrich CVEs with details that help prioritize patching and mitigation efforts, as the NVD backlog of unenriched CVEs awaiting analysis surpasses 10,000.
Security News
Socket is joining forces with CISA and other industry leaders at the RSA Conference to sign the Secure by Design pledge, committing to uphold the highest security standards in our products.
Research
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
