Socket
Socket
Sign inDemoInstall

formidable

Package Overview
Dependencies
4
Maintainers
10
Versions
78
Alerts
File Explorer

Advanced tools

Install Socket

Detect and block malicious and high-risk dependencies

Install

Comparing version 2.0.0-canary.20200129.32 to 2.0.0-dev.20200130.1

src/parsers/index.js

2

CHANGELOG.md

@@ -24,2 +24,4 @@

* fix: update docs and examples [#544](https://github.com/node-formidable/node-formidable/pull/544) ([#248](https://github.com/node-formidable/node-formidable/issues/248), [#335](https://github.com/node-formidable/node-formidable/issues/335), [#371](https://github.com/node-formidable/node-formidable/issues/371), [#372](https://github.com/node-formidable/node-formidable/issues/372), [#387](https://github.com/node-formidable/node-formidable/issues/387), partly [#471](https://github.com/node-formidable/node-formidable/issues/471), [#535](https://github.com/node-formidable/node-formidable/issues/535))
* feat: introduce Plugins API, fix silent failing tests ([#545](https://github.com/node-formidable/node-formidable/pull/545), [#391](https://github.com/node-formidable/node-formidable/pull/391), [#407](https://github.com/node-formidable/node-formidable/pull/407), [#386](https://github.com/node-formidable/node-formidable/pull/386), [#374](https://github.com/node-formidable/node-formidable/pull/374), [#521](https://github.com/node-formidable/node-formidable/pull/521), [#267](https://github.com/node-formidable/node-formidable/pull/267))
* respect form hash option on incoming octect/stream requests ([#407](https://github.com/node-formidable/node-formidable/pull/407))
* fix: exposing file writable stream errors ([#520](https://github.com/node-formidable/node-formidable/pull/520), [#316](https://github.com/node-formidable/node-formidable/pull/316), [#469](https://github.com/node-formidable/node-formidable/pull/469), [#470](https://github.com/node-formidable/node-formidable/pull/470))

@@ -26,0 +28,0 @@

14

package.json
{
"name": "formidable",
"version": "2.0.0-canary.20200129.32",
"version": "2.0.0-dev.20200130.1",
"license": "MIT",

@@ -11,7 +11,8 @@ "description": "A node.js module for parsing form data, especially file uploads.",

"files": [
"src"
"src",
"test"
],
"publishConfig": {
"access": "public",
"tag": "canary"
"tag": "dev"
},

@@ -27,2 +28,6 @@ "scripts": {

},
"dependencies": {
"dezalgo": "^1.0.3",
"once": "^1.4.0"
},
"devDependencies": {

@@ -35,2 +40,4 @@ "@tunnckocore/prettier-config": "^1.2.0",

"eslint-plugin-prettier": "^3.1.2",
"jest": "^25.1.0",
"koa": "^2.11.0",
"nyc": "^15.0.0",

@@ -40,2 +47,3 @@ "prettier": "^1.19.1",

"request": "^2.88.0",
"supertest": "^4.0.2",
"urun": "^0.0.8",

@@ -42,0 +50,0 @@ "utest": "^0.0.8"

@@ -174,2 +174,3 @@ <p align="center">

- `options.multiples` **{boolean}** - default `false`; when you call the `.parse` method, the `files` argument (of the callback) will contain arrays of files for inputs which submit multiple files using the HTML5 `multiple` attribute. Also, the `fields` argument will contain arrays of values for fields that have names ending with '[]'.
- `options.enabledPlugins` **{array}** - default `['octetstream', 'urlencoded', 'multipart', 'json']`; list of enabled built-in plugins, for custom ones use `form.use(() => {})` before the `.parse()` method, see [plugins](#plugins)

@@ -258,2 +259,63 @@ _**Note:** If this value is exceeded, an `'error'` event is emitted._

### .use(plugin: Plugin)
A method that allows you to extend the Formidable library. By default we include 4 plugins,
which esentially are adapters to plug the different built-in parsers.
**The plugins added by this method are always enabled.**
_See [src/plugins/](./src/plugins/) for more detailed look on default plugins._
The `plugin` param has such signature:
```typescript
function(formidable: Formidable, options: Options): void;
```
The architecture is simple. The `plugin` is a function that is passed with
the Formidable instance (the `form` across the README examples) and the options.
**Note:** the plugin function's `this` context is also the same instance.
```js
const formidable = require('formidable');
const form = formidable({ keepExtensions: true });
form.use((self, options) => {
// self === this === form
console.log('woohoo, custom plugin');
// do your stuff; check `src/plugins` for inspiration
});
form.parse(req, (error, fields, files) => {
console.log('done!');
});
```
**Important to note**, is that inside plugin `this.options`, `self.options` and `options`
MAY or MAY NOT be the same. General best practice is to always use the `this`, so you can
later test your plugin independently and more easily.
If you want to disable some parsing capabilities of Formidable, you can disable the plugin
which corresponds to the parser. For example, if you want to disable multipart parsing
(so the [src/parsers/Multipart.js](./src/parsers/Multipart.js) which is used in [src/plugins/multipart.js](./src/plugins/multipart.js)), then you can remove it from
the `options.enabledPlugins`, like so
```js
const { Formidable } = require('formidable');
const form = new Formidable({
hash: 'sha1',
enabledPlugins: ['octetstream', 'querystring', 'json'],
});
```
**Be aware** that the order _MAY_ be important too. The names corresponds 1:1
to files in [src/plugins/](./src/plugins) folder.
Pull requests for new built-in plugins MAY be accepted - for example,
more advanced querystring parser. Add your plugin as a new file
in `src/plugins/` folder (lowercased) and follow how the other plugins are made.
### form.onPart

@@ -419,2 +481,3 @@

<td align="center"><a href="https://github.com/gabipetrovay"><img src="https://avatars0.githubusercontent.com/u/1170398?v=4" width="100px;" alt=""/><br /><sub><b>Gabriel Petrovay</b></sub></a><br /><a href="https://github.com/node-formidable/node-formidable/issues?q=author%3Agabipetrovay" title="Bug reports">🐛</a> <a href="https://github.com/node-formidable/node-formidable/commits?author=gabipetrovay" title="Code">💻</a></td>
<td align="center"><a href="https://github.com/Elzair"><img src="https://avatars0.githubusercontent.com/u/2352818?v=4" width="100px;" alt=""/><br /><sub><b>Philip Woods</b></sub></a><br /><a href="https://github.com/node-formidable/node-formidable/commits?author=Elzair" title="Code">💻</a> <a href="#ideas-Elzair" title="Ideas, Planning, & Feedback">🤔</a></td>
</tr>

@@ -425,2 +488,3 @@ </table>

<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->

@@ -427,0 +491,0 @@

@@ -10,3 +10,4 @@ /* eslint-disable class-methods-use-this */

const crypto = require('crypto');
const { Stream } = require('stream');
const once = require('once');
const dezalgo = require('dezalgo');
const { EventEmitter } = require('events');

@@ -23,12 +24,8 @@ const { StringDecoder } = require('string_decoder');

multiples: false,
enabledPlugins: ['octetstream', 'querystring', 'multipart', 'json'],
};
const File = require('./File');
/** Parsers */
const JSONParser = require('./parsers/JSON');
const DummyParser = require('./parsers/Dummy');
const OctetParser = require('./parsers/OctetStream');
const MultipartParser = require('./parsers/Multipart');
const QuerystringParser = require('./parsers/Querystring');

@@ -45,3 +42,3 @@ function hasOwnProp(obj, key) {

Object.assign(this, DEFAULT_OPTIONS, options);
this.options = { ...DEFAULT_OPTIONS, ...options };
this.uploadDir = this.uploadDir || os.tmpdir();

@@ -59,5 +56,30 @@

this._fileSize = 0;
this._plugins = [];
this.openedFiles = [];
const enabledPlugins = []
.concat(this.options.enabledPlugins)
.filter(Boolean);
if (enabledPlugins.length === 0) {
throw new Error(
'expect at least 1 enabled builtin plugin, see options.enabledPlugins',
);
}
this.options.enabledPlugins.forEach((pluginName) => {
const plgName = pluginName.toLowerCase();
// eslint-disable-next-line import/no-dynamic-require, global-require
this.use(require(path.join(__dirname, 'plugins', `${plgName}.js`)));
});
}
use(plugin) {
if (typeof plugin !== 'function') {
throw new Error('.use: expect `plugin` to be a function');
}
this._plugins.push(plugin.bind(this));
return this;
}
parse(req, cb) {

@@ -95,2 +117,3 @@ this.pause = () => {

if (cb) {
const callback = once(dezalgo(cb));
const fields = {};

@@ -101,3 +124,3 @@ const files = {};

// TODO: too much nesting
if (this.multiples && name.slice(-2) === '[]') {
if (this.options.multiples && name.slice(-2) === '[]') {
const realName = name.slice(0, name.length - 2);

@@ -122,3 +145,3 @@ if (hasOwnProp(fields, realName)) {

// TODO: too much nesting
if (this.multiples) {
if (this.options.multiples) {
if (hasOwnProp(files, name)) {

@@ -142,6 +165,6 @@ if (!Array.isArray(files[name])) {

this.on('error', (err) => {
cb(err, fields, files);
callback(err, fields, files);
});
this.on('end', () => {
cb(null, fields, files);
callback(null, fields, files);
});

@@ -173,4 +196,6 @@ }

}
this._parser.end();
if (this._parser) {
this._parser.end();
}
this._maybeEnd();
});

@@ -185,2 +210,8 @@

this._parseContentType();
if (!this._parser) {
this._error(new Error('not parser found'));
return;
}
this._parser.once('error', (error) => {

@@ -220,6 +251,6 @@ this._error(error);

// this method can be overwritten by the user
this.handlePart(part);
this._handlePart(part);
}
handlePart(part) {
_handlePart(part) {
if (part.filename && typeof part.filename !== 'string') {

@@ -242,10 +273,12 @@ this._error(new Error(`the part.filename should be string when exists`));

let value = '';
const decoder = new StringDecoder(part.transferEncoding || this.encoding);
const decoder = new StringDecoder(
part.transferEncoding || this.options.encoding,
);
part.on('data', (buffer) => {
this._fieldsSize += buffer.length;
if (this._fieldsSize > this.maxFieldsSize) {
if (this._fieldsSize > this.options.maxFieldsSize) {
this._error(
new Error(
`maxFieldsSize exceeded, received ${this._fieldsSize} bytes of field data`,
`options.maxFieldsSize exceeded, received ${this._fieldsSize} bytes of field data`,
),

@@ -270,3 +303,3 @@ );

type: part.mime,
hash: this.hash,
hash: this.options.hash,
});

@@ -283,6 +316,6 @@ file.on('error', (err) => {

this._fileSize += buffer.length;
if (this._fileSize > this.maxFileSize) {
if (this._fileSize > this.options.maxFileSize) {
this._error(
new Error(
`maxFileSize exceeded, received ${this._fileSize} bytes of file data`,
`options.maxFileSize exceeded, received ${this._fileSize} bytes of file data`,
),

@@ -313,3 +346,3 @@ );

if (this.bytesExpected === 0) {
this._parser = new DummyParser(this);
this._parser = new DummyParser(this, this.options);
return;

@@ -323,39 +356,47 @@ }

if (this.headers['content-type'].match(/octet-stream/i)) {
this._initOctetStream();
return;
}
const results = [];
const _dummyParser = new DummyParser(this, this.options);
if (this.headers['content-type'].match(/urlencoded/i)) {
this._initUrlencoded();
return;
}
// eslint-disable-next-line no-plusplus
for (let idx = 0; idx < this._plugins.length; idx++) {
const plugin = this._plugins[idx];
if (this.headers['content-type'].match(/multipart/i)) {
const m = this.headers['content-type'].match(
/boundary=(?:"([^"]+)"|([^;]+))/i,
);
if (m) {
this._initMultipart(m[1] || m[2]);
} else {
this._error(
new Error('bad content-type header, no multipart boundary'),
let pluginReturn = null;
try {
pluginReturn = plugin(this, this.options) || this;
} catch (err) {
// directly throw from the `form.parse` method;
// there is no other better way, except a handle through options
const error = new Error(
`plugin on index ${idx} failed with: ${err.message}`,
);
error.idx = idx;
throw error;
}
return;
}
if (this.headers['content-type'].match(/json/i)) {
this._initJSONencoded();
return;
Object.assign(this, pluginReturn);
// todo: use Set/Map and pass plugin name instead of the `idx` index
this.emit('plugin', idx, pluginReturn);
results.push(pluginReturn);
}
this._error(
new Error(
`bad content-type header, unknown content-type: ${this.headers['content-type']}`,
),
);
this.emit('pluginsResults', results);
// ? probably not needed, because we check options.enabledPlugins in the constructor
// if (results.length === 0 /* && results.length !== this._plugins.length */) {
// this._error(
// new Error(
// `bad content-type header, unknown content-type: ${this.headers['content-type']}`,
// ),
// );
// }
}
_error(err) {
_error(err, eventName = 'error') {
// if (!err && this.error) {
// this.emit('error', this.error);
// return;
// }
if (this.error || this.ended) {

@@ -366,3 +407,3 @@ return;

this.error = err;
this.emit('error', err);
this.emit(eventName, err);

@@ -391,129 +432,6 @@ if (Array.isArray(this.openedFiles)) {

_newParser() {
return new MultipartParser();
return new MultipartParser(this.options);
}
_initMultipart(boundary) {
this.type = 'multipart';
const parser = new MultipartParser();
let headerField;
let headerValue;
let part;
parser.initWithBoundary(boundary);
// eslint-disable-next-line max-statements, consistent-return
parser.on('data', ({ name, buffer, start, end }) => {
if (name === 'partBegin') {
part = new Stream();
part.readable = true;
part.headers = {};
part.name = null;
part.filename = null;
part.mime = null;
part.transferEncoding = 'binary';
part.transferBuffer = '';
headerField = '';
headerValue = '';
} else if (name === 'headerField') {
headerField += buffer.toString(this.encoding, start, end);
} else if (name === 'headerValue') {
headerValue += buffer.toString(this.encoding, start, end);
} else if (name === 'headerEnd') {
headerField = headerField.toLowerCase();
part.headers[headerField] = headerValue;
// matches either a quoted-string or a token (RFC 2616 section 19.5.1)
const m = headerValue.match(
// eslint-disable-next-line no-useless-escape
/\bname=("([^"]*)"|([^\(\)<>@,;:\\"\/\[\]\?=\{\}\s\t/]+))/i,
);
if (headerField === 'content-disposition') {
if (m) {
part.name = m[2] || m[3] || '';
}
part.filename = this._fileName(headerValue);
} else if (headerField === 'content-type') {
part.mime = headerValue;
} else if (headerField === 'content-transfer-encoding') {
part.transferEncoding = headerValue.toLowerCase();
}
headerField = '';
headerValue = '';
} else if (name === 'headersEnd') {
switch (part.transferEncoding) {
case 'binary':
case '7bit':
case '8bit': {
const dataPropagation = (ctx) => {
if (ctx.name === 'partData') {
part.emit('data', ctx.buffer.slice(ctx.start, ctx.end));
}
};
const dataStopPropagation = (ctx) => {
if (ctx.name === 'partEnd') {
part.emit('end');
parser.off('data', dataPropagation);
parser.off('data', dataStopPropagation);
}
};
parser.on('data', dataPropagation);
parser.on('data', dataStopPropagation);
break;
}
case 'base64': {
const dataPropagation = (ctx) => {
if (ctx.name === 'partData') {
part.transferBuffer += ctx.buffer
.slice(ctx.start, ctx.end)
.toString('ascii');
/*
four bytes (chars) in base64 converts to three bytes in binary
encoding. So we should always work with a number of bytes that
can be divided by 4, it will result in a number of buytes that
can be divided vy 3.
*/
const offset = parseInt(part.transferBuffer.length / 4, 10) * 4;
part.emit(
'data',
Buffer.from(
part.transferBuffer.substring(0, offset),
'base64',
),
);
part.transferBuffer = part.transferBuffer.substring(offset);
}
};
const dataStopPropagation = (ctx) => {
if (ctx.name === 'partEnd') {
part.emit('data', Buffer.from(part.transferBuffer, 'base64'));
part.emit('end');
parser.off('data', dataPropagation);
parser.off('data', dataStopPropagation);
}
};
parser.on('data', dataPropagation);
parser.on('data', dataStopPropagation);
break;
}
default:
return this._error(new Error('unknown transfer-encoding'));
}
this.onPart(part);
} else if (name === 'end') {
this.ended = true;
this._maybeEnd();
}
});
this._parser = parser;
}
_fileName(headerValue) {
_getFileName(headerValue) {
// matches either a quoted-string or a token (RFC 2616 section 19.5.1)

@@ -535,94 +453,2 @@ const m = headerValue.match(

_initUrlencoded() {
this.type = 'urlencoded';
const parser = new QuerystringParser(this.maxFields);
parser.on('data', ({ key, value }) => {
this.emit('field', key, value);
});
parser.once('end', () => {
this.ended = true;
this._maybeEnd();
});
this._parser = parser;
}
_initOctetStream() {
this.type = 'octet-stream';
const filename = this.headers['x-file-name'];
const mime = this.headers['content-type'];
const file = new File({
path: this._uploadPath(filename),
name: filename,
type: mime,
});
file.on('error', (err) => {
this._error(err);
});
this.emit('fileBegin', filename, file);
file.open();
this.openedFiles.push(file);
this._flushing += 1;
this._parser = new OctetParser();
// Keep track of writes that haven't finished so we don't emit the file before it's done being written
let outstandingWrites = 0;
this._parser.on('data', (buffer) => {
this.pause();
outstandingWrites += 1;
file.write(buffer, () => {
outstandingWrites -= 1;
this.resume();
if (this.ended) {
this._parser.emit('doneWritingFile');
}
});
});
this._parser.on('end', () => {
this._flushing -= 1;
this.ended = true;
const done = () => {
file.end(() => {
this.emit('file', 'file', file);
this._maybeEnd();
});
};
if (outstandingWrites === 0) {
done();
} else {
this._parser.once('doneWritingFile', done);
}
});
}
_initJSONencoded() {
this.type = 'json';
const parser = new JSONParser();
parser.on('data', ({ key, value }) => {
this.emit('field', key, value);
});
parser.once('end', () => {
this.ended = true;
this._maybeEnd();
});
this._parser = parser;
}
_uploadPath(filename) {

@@ -632,3 +458,3 @@ const buf = crypto.randomBytes(16);

if (this.keepExtensions) {
if (this.options.keepExtensions) {
let ext = path.extname(filename);

@@ -655,2 +481,3 @@ ext = ext.replace(/(\.[a-z0-9]+).*/i, '$1');

IncomingForm.DEFAULT_OPTIONS = DEFAULT_OPTIONS;
module.exports = IncomingForm;

@@ -6,7 +6,4 @@ 'use strict';

const JSONParser = require('./parsers/JSON');
const DummyParser = require('./parsers/Dummy');
const MultipartParser = require('./parsers/Multipart');
const OctetStreamParser = require('./parsers/OctetStream');
const QuerystringParser = require('./parsers/Querystring');
const plugins = require('./plugins/index');
const parsers = require('./parsers/index');

@@ -26,11 +23,13 @@ // make it available without requiring the `new` keyword

// parsers
JSONParser,
DummyParser,
MultipartParser,
OctetStreamParser,
QuerystringParser,
...parsers,
parsers,
// typo aliases
OctetstreamParser: OctetStreamParser,
QueryStringParser: QuerystringParser,
// misc
defaultOptions: Formidable.DEFAULT_OPTIONS,
enabledPlugins: Formidable.DEFAULT_OPTIONS.enabledPlugins,
// plugins
plugins: {
...plugins,
},
});

@@ -8,4 +8,5 @@ /* eslint-disable no-underscore-dangle */

class DummyParser extends Transform {
constructor(incomingForm) {
constructor(incomingForm, options = {}) {
super();
this.globalOptions = { ...options };
this.incomingForm = incomingForm;

@@ -12,0 +13,0 @@ }

@@ -8,5 +8,6 @@ /* eslint-disable no-underscore-dangle */

class JSONParser extends Transform {
constructor() {
constructor(options = {}) {
super({ readableObjectMode: true });
this.chunks = [];
this.globalOptions = { ...options };
}

@@ -13,0 +14,0 @@

@@ -46,3 +46,3 @@ /* eslint-disable no-fallthrough */

class MultipartParser extends Transform {
constructor() {
constructor(options = {}) {
super({ readableObjectMode: true });

@@ -55,2 +55,3 @@ this.boundary = null;

this.globalOptions = { ...options };
this.index = null;

@@ -57,0 +58,0 @@ this.flags = 0;

@@ -5,4 +5,9 @@ 'use strict';

class OctetStreamParser extends PassThrough {}
class OctetStreamParser extends PassThrough {
constructor(options = {}) {
super();
this.globalOptions = { ...options };
}
}
module.exports = OctetStreamParser;

@@ -11,5 +11,6 @@ /* eslint-disable no-underscore-dangle */

class QuerystringParser extends Transform {
constructor(maxKeys) {
constructor(options = {}) {
super({ readableObjectMode: true });
this.maxKeys = maxKeys;
this.globalOptions = { ...options };
this.maxKeys = this.globalOptions.maxFields;
this.buffer = '';

@@ -16,0 +17,0 @@ this.bufferLength = 0;

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc