jsonrepair
Advanced tools
Comparing version 0.1.0 to 1.1.0
{ | ||
"name": "jsonrepair", | ||
"description": "Repair a malformed JSON stream", | ||
"version": "0.1.0", | ||
"main" : "index.js", | ||
"directories" : { | ||
"example" : "example" | ||
}, | ||
"author": { | ||
"name": "Volker Mische", | ||
"email": "volker.mische@gmail.com", | ||
"url": "http://vmx.cx/" | ||
}, | ||
"homepage": "https://github.com/vmx/jsonrepair", | ||
"repository": { | ||
"type": "git", | ||
"url": "git://github.com/vmx/jsonrepair.git" | ||
}, | ||
"bugs": "https://github.com/vmx/jsonrepair/issues", | ||
"keywords": [ | ||
"JSON", | ||
"stream", | ||
"streaming", | ||
"repair", | ||
"malformed" | ||
], | ||
"licenses" : [ | ||
{ | ||
"type": "MIT", | ||
"url": "http://github.com/vmx/raw/master/LICENSE" | ||
} | ||
], | ||
"dependencies": { | ||
"clarinet" : "~0.7.0" | ||
} | ||
"name": "jsonrepair", | ||
"version": "1.1.0", | ||
"description": "Repair broken JSON documents", | ||
"repository": { | ||
"type": "git", | ||
"url": "https://github.com/josdejong/simple-json-repair.git" | ||
}, | ||
"type": "module", | ||
"main": "lib/cjs/index-commonjs.js", | ||
"module": "lib/esm/simpleJsonRepair.js", | ||
"browser": "lib/umd/simpleJsonRepair.min.js", | ||
"types": "lib/esm/simpleJsonRepair.d.ts", | ||
"keywords": [ | ||
"simple", | ||
"json", | ||
"repair", | ||
"fix", | ||
"invalid" | ||
], | ||
"scripts": { | ||
"test": "mocha", | ||
"test:lib": "mocha test-lib/*.test.*", | ||
"build": "npm run clean && npm run build:esm && npm run build:esm:fix:imports && npm run build:cjs && npm run build:umd && npm run build:umd:min", | ||
"clean": "del-cli lib", | ||
"build:esm": "tsc --module es2015 --target es5 --outDir lib/esm", | ||
"build:esm:fix:imports": "node tools/esm/fixImportExtensions.cjs", | ||
"build:cjs": "tsc --module commonjs --target es5 --outDir lib/cjs && cp-cli tools/cjs lib/cjs", | ||
"build:umd": "rollup lib/esm/simpleJsonRepair.js --format umd --name 'simpleJsonRepair' --sourcemap --output.file lib/umd/simpleJsonRepair.js && cp-cli tools/cjs/package.json lib/umd/package.json", | ||
"build:umd:min": "uglifyjs --compress --mangle --source-map --comments --output lib/umd/simpleJsonRepair.min.js -- lib/umd/simpleJsonRepair.js", | ||
"link": "npm run build", | ||
"lint": "eslint src/**/*.ts", | ||
"build-and-test": "npm run lint && npm run build && npm run test:lib", | ||
"prepublishOnly": "npm run build-and-test" | ||
}, | ||
"files": [ | ||
"README.md", | ||
"LICENSE.md", | ||
"lib" | ||
], | ||
"author": "Jos de Jong", | ||
"license": "ISC", | ||
"devDependencies": { | ||
"@types/mocha": "8.0.3", | ||
"@typescript-eslint/eslint-plugin": "4.6.0", | ||
"@typescript-eslint/parser": "4.6.0", | ||
"cp-cli": "2.0.0", | ||
"del-cli": "3.0.1", | ||
"eslint": "7.12.1", | ||
"eslint-config-standard": "16.0.1", | ||
"eslint-plugin-import": "2.22.1", | ||
"eslint-plugin-node": "11.1.0", | ||
"eslint-plugin-promise": "4.2.1", | ||
"mocha": "8.2.0", | ||
"rollup": "2.33.1", | ||
"ts-node": "9.0.0", | ||
"tslib": "2.0.3", | ||
"typescript": "4.0.5", | ||
"uglify-js": "3.11.5" | ||
} | ||
} |
120
README.md
@@ -1,72 +0,88 @@ | ||
jsonrepair | ||
========== | ||
# simple-json-repair | ||
jsonrepair is transforming malformed JSON streams into valid ones. It uses [clarinet](https://github.com/dscape/clarinet/) in the background to do the JSON parsing. Whenever an error occurs, you can handle that and move and with parsing. This is useful when your JSON got automatically generated from some other source, but contains some errors. Things like trailing commas in arrays, or two subsequent commas. | ||
Repair invalid JSON documents. | ||
Try it out: https://josdejong.github.io/simple-json-repair/ | ||
Defining rules | ||
-------------- | ||
The following issues can be fixed: | ||
To transform the malformed JSON, you need to define rules that should be applied whenever a parsing error occurs. For example when you want to remove a trailing comma in an array, if there is any, the rule would look like this: | ||
- Add missing quotes around keys | ||
- Replace single quotes with double quotes | ||
- Turn newline delimited JSON into a valid JSON array | ||
- Add missing escape characters | ||
- Replace special white space characters with regular spaces | ||
- Replace special quote characters like `“...”` with regular double quotes | ||
- Concatenate strings like `"long text" + "more text on next line"` | ||
- Add missing commas | ||
- Strip trailing commas | ||
- Strip comments like `/* ... */` and `// ...` | ||
- Strip JSONP notation like `callback({ ... })` | ||
- Strip MongoDB data types like `NumberLong(2)` and `ISODate("2012-12-19T06:01:17.171Z")` | ||
- Replace Python constants `None`, `True`, and `False` with `null`, `true`, and `false` | ||
{ | ||
description: 'Removing trailing comma within array', | ||
character: ']', | ||
expected: 'VALUE', | ||
action: function(parser) { | ||
parser.stack.pop(); | ||
parser.onclosearray(); | ||
var newState = parser.stack.pop(); | ||
parser.state = newState; | ||
} | ||
} | ||
The `character` defines the character that is currently processed. In case of the trailing comma case, we see an error when the closing bracket `]` is hit. | ||
## Install | ||
The `expected` property defines in which state the error is happening. When a comma within an array was read, the next state is `VALUE` as it expects to read the next value. Here's a list of possible values for the `expected` property: | ||
``` | ||
$ npm install simple-json-repair | ||
``` | ||
- **BEGIN**: Something in front of everything (e.g. a header) | ||
- **OPEN_OBJECT**: Right after the opening bracket of an object | ||
- **CLOSE_OBJECT**: After the value of a key-value pair (before the comma or the ending bracket) | ||
- **OPEN_KEY**: Right before the key of an object (after a comma or the opening bracket) | ||
- **CLOSE_KEY**: Right after the key, *before* the colon | ||
- **VALUE**: | ||
- In an object: After the key, before the value (after the colon) | ||
- In an array: After a previous value (after the comma) or right after the opening bracket | ||
- **OPEN_ARRAY**: Right after the opening bracket of an array | ||
- **CLOSE_ARRAY**: After the value in an array (before the comma or the ending bracket) | ||
Note that in the `lib` folder, there are builds for ESM, UMD, and CommonJs. | ||
`action` defines what should happen. It's a function that takes the internal clarinet parser as an input. There you can manipulate the current state of the parser to make it move on in the expected way. | ||
## Use | ||
Full example | ||
------------ | ||
```js | ||
import simpleJsonRepair from 'simple-json-repair' | ||
var fs = require('fs'); | ||
var RepairStream = require('jsonrepair').RepairStream; | ||
// The following is invalid JSON: is consists of JSON contents copied from | ||
// a JavaScript code base, where the keys are missing double quotes, | ||
// and strings are using single quotes: | ||
const json = '{name: \'John\'}' | ||
var repairRules = [{ | ||
description: 'Removing trailing comma within array', | ||
character: ']', | ||
expected: 'VALUE', | ||
action: function(parser) { | ||
parser.stack.pop(); | ||
parser.onclosearray(); | ||
var newState = parser.stack.pop(); | ||
parser.state = newState; | ||
} | ||
}]; | ||
const repaired = simpleJsonRepair(json) | ||
console.log(repaired) // '{"name": "John"}' | ||
``` | ||
var input = fs.createReadStream('malformed.json'); | ||
var repairJson = new RepairStream(repairRules); | ||
input.pipe(repairJson).pipe(process.stdout); | ||
### API | ||
``` | ||
simpleJsonRepair(json: string) : string | ||
``` | ||
See the `example/` directory for more examples. | ||
The function `simpleJsonRepair` throws an exception when an issue is encountered | ||
which could not be solved. When no error is thrown, the output will be valid JSON. | ||
License | ||
------- | ||
### Develop | ||
The code is licensed under the MIT License. | ||
To build the library (ESM, CommonJs, and UMD output in the folder `lib`): | ||
``` | ||
$ npm install | ||
$ npm run build | ||
``` | ||
To run the unit tests: | ||
``` | ||
$ npm test | ||
``` | ||
To run the linter (eslint): | ||
``` | ||
$ npm run lint | ||
``` | ||
To run the linter, build all, and run unit tests and integration tests: | ||
``` | ||
$ npm run build-and-test | ||
``` | ||
## License | ||
Released under the [ISC license](LICENSE.md). |
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Major refactor
Supply chain riskPackage has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.
Found 1 instance in 1 package
New author
Supply chain riskA new npm collaborator published a version of the package for the first time. New collaborators are usually benign additions to a project, but do indicate a change to the security surface area of a package.
Found 1 instance in 1 package
No bug tracker
MaintenancePackage does not have a linked bug tracker in package.json.
Found 1 instance in 1 package
No website
QualityPackage does not have a website.
Found 1 instance in 1 package
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Filesystem access
Supply chain riskAccesses the file system, and could potentially read sensitive data.
Found 1 instance in 1 package
No v1
QualityPackage is not semver >=1. This means it is not stable and does not support ^ ranges.
Found 1 instance in 1 package
129118
0
23
2144
89
0
Yes
16
2
- Removedclarinet@~0.7.0
- Removedclarinet@0.7.3(transitive)