@diotoborg/quos-accusantium
Parse JSON without risk of losing numeric information.
import { parse, stringify } from '@diotoborg/quos-accusantium'
const text = '{"decimal":2.370,"long":9123372036854000123,"big":2.3e+500}'
console.log(JSON.stringify(JSON.parse(text)))
console.log(stringify(parse(text)))
The following in-depth article explains what happens there: Why does JSON.parse corrupt large numbers and how to solve this?
How does it work? The library works exactly the same as the native JSON.parse
and JSON.stringify
. The difference is that @diotoborg/quos-accusantium
preserves information of big numbers. @diotoborg/quos-accusantium
parses numeric values not as a regular number but as a LosslessNumber
, a lightweight class which stores the numeric value as a string. One can perform regular operations with a LosslessNumber
, and it will throw an error when this would result in losing information.
When to use? If you have to deal with JSON data that contains long
values for example, coming from an application like C++, Java, or C#. The trade-off is that @diotoborg/quos-accusantium
is slower than the native JSON.parse
and JSON.stringify
functions, so be careful when performance is a bottleneck for you.
Features:
- No risk of losing numeric information when working with big numbers.
- Maintain the formatting of numbers.
- Parse error on duplicate keys.
- Built-in support for
bigint
. - Built-in support for
Date
(turned off by default). - Customizable: parse numeric values into any data type, like
BigNumber
, bigint
, number
, or a mix of them. - Compatible with the native, built-in
JSON.parse
and JSON.stringify
. - Helpful error messages when parsing invalid JSON.
- Works in browsers and node.js.
- Comes with TypeScript typings included.
- Modular: ES module functions, only load and bundle what you use.
- The full bundle is less than 4kB in size when minified and gzipped.
Install
Install via npm:
npm install @diotoborg/quos-accusantium
Use
Parse and stringify
Parsing and stringification works as you're used to:
import { parse, stringify } from '@diotoborg/quos-accusantium'
const json = parse('{"foo":"bar"}')
const text = stringify(json)
LosslessNumbers
Numbers are parsed into a LosslessNumber
, which can be used like a regular number in numeric operations. Converting to a number will throw an error when this would result in losing information due to truncation, overflow, or underflow.
import { parse } from '@diotoborg/quos-accusantium'
const text = '{"normal":2.3,"long":123456789012345678901,"big":2.3e+500}'
const json = parse(text)
console.log(json.normal.isLosslessNumber)
console.log(json.normal.valueOf())
console.log(json.normal + 2)
console.log(json.long + 1)
BigInt
JavaScript natively supports bigint
: big integers that can hold a large number of digits, instead of the about 15 digits that a regular number
can hold. It is a typical use case to want to parse integer numbers into a bigint
, and all other values into a regular number
. This can be achieved with a custom numberParser
:
import { parse, isInteger } from '@diotoborg/quos-accusantium'
export function customNumberParser(value) {
return isInteger(value) ? BigInt(value) : parseFloat(value)
}
const text = '[123456789123456789123456789, 2.3, 123]'
const json = parse(text, null, customNumberParser)
You can adjust the logic to your liking, using utility functions like isInteger
, isNumber
, isSafeNumber
. The number parser shown above is included in the library and is named parseNumberAndBigInt
.
Validate safe numbers
If you want parse a json string into an object with regular numbers, but want to validate that no numeric information is lost, you write your own number parser and use isSafeNumber
to validate the numbers:
import { parse, isSafeNumber } from '@diotoborg/quos-accusantium'
function parseAndValidateNumber(value) {
if (!isSafeNumber(value)) {
throw new Error(`Cannot safely convert value '${value}' into a number`)
}
return parseFloat(value)
}
let json = parse('[1,2,3]', undefined, parseAndValidateNumber)
console.log(json)
try {
let json = parse('[1,2e+500,3]', undefined, parseAndValidateNumber)
} catch (err) {
console.log(err)
}
BigNumbers
To use the library in conjunction with your favorite BigNumber library, for example decimal.js. You have to define a custom number parser and stringifier:
import { parse, stringify } from '@diotoborg/quos-accusantium'
import Decimal from 'decimal.js'
const parseDecimal = (value) => new Decimal(value)
const decimalStringifier = {
test: (value) => Decimal.isDecimal(value),
stringify: (value) => value.toString()
}
const text = '{"value":2.3e500}'
const json = parse(text, undefined, parseDecimal)
const output = {
result: json.value.times(2)
}
const str = stringify(output, undefined, undefined, [decimalStringifier])
Reviver and replacer
The library is compatible with the native JSON.parse
and JSON.stringify
, and also comes with the optional reviver
and replacer
arguments that allow you to serialize for example data classes in a custom way. Here is an example demonstrating how you can stringify a Date
in a different way than the built-in reviveDate
utility function.
The following example stringifies a Date
as an object with a $date
key instead of a string, so it is uniquely recognizable when parsing the structure:
import { parse, stringify } from '@diotoborg/quos-accusantium'
function customDateReplacer(key, value) {
if (value instanceof Date) {
return {
$date: value.toISOString()
}
}
return value
}
function isJSONDateObject(value) {
return value && typeof value === 'object' && typeof value.$date === 'string'
}
function customDateReviver(key, value) {
if (isJSONDateObject(value)) {
return new Date(value.$date)
}
return value
}
const record = {
message: 'Hello World',
timestamp: new Date('2022-08-30T09:00:00Z')
}
const text = stringify(record, customDateReplacer)
console.log(text)
const parsed = parse(text, customDateReviver)
console.log(parsed)
API
parse(text [, reviver [, parseNumber]])
The LosslessJSON.parse()
function parses a string as JSON, optionally transforming the value produced by parsing.
- @param
{string} text
The string to parse as JSON. See the JSON object for a description of JSON syntax. - @param
{(key: string, value: unknown) => unknown} [reviver]
If a function, prescribes how the value originally produced by parsing is transformed, before being returned. - @param
{function(value: string) : unknown} [parseNumber]
Pass an optional custom number parser. Input is a string, and the output can be any numeric value: number
, bigint
, LosslessNumber
, or a custom BigNumber
library. By default, all numeric values are parsed into a LosslessNumber
. - @returns
{unknown}
Returns the Object corresponding to the given JSON text. - @throws Throws a SyntaxError exception if the string to parse is not valid JSON.
stringify(value [, replacer [, space [, numberStringifiers]]])
The LosslessJSON.stringify()
function converts a JavaScript value to a JSON string, optionally replacing values if a replacer function is specified, or optionally including only the specified properties if a replacer array is specified.
- @param
{unknown} value
The value to convert to a JSON string. - @param
{((key: string, value: unknown) => unknown) | Array.<string | number>} [replacer]
A function that alters the behavior of the stringification process, or an array with strings or numbers that serve as a whitelist for selecting the properties of the value object to be included in the JSON string. If this value is null
or not provided, all properties of the object are included in the resulting JSON string. - @param
{number | string | undefined} [space]
A string
or number
that is used to insert white space into the output JSON string for readability purposes. If this is a number
, it indicates the number of space characters to use as white space. Values less than 1 indicate that no space should be used. If this is a string
, the string
is used as white space. If this parameter is not provided (or is null
), no white space is used. - @param
{Array<{test: (value: unknown) => boolean, stringify: (value: unknown) => string}>} [numberStringifiers]
An optional list with additional number stringifiers, for example to serialize a BigNumber
. The output of the function must be valid stringified JSON number. When undefined
is returned, the property will be deleted from the object. The difference with using a replacer
is that the output of a replacer
must be JSON and will be stringified afterwards, whereas the output of the numberStringifiers
is already stringified JSON. - @returns
{string | undefined}
Returns the string representation of the JSON object. - @throws Throws a SyntaxError when one of the
numberStringifiers
does not return valid output.
LosslessNumber
Construction
new LosslessNumber(value: number | string) : LosslessNumber
Methods
-
.valueOf(): number | bigint
Convert the LosslessNumber
into a regular number
or bigint
. A number
is returned for safe numbers and decimal values that only lose some insignificant digits. A bigint
is returned for large integer numbers. An Error
is thrown for values that will overflow or underflow. Examples:
console.log(new LosslessNumber('23.4').valueOf())
console.log(new LosslessNumber('0.66666666666666666666667').valueOf())
console.log(new LosslessNumber('9123372036854000123').valueOf())
console.log(new LosslessNumber('2.3e+500').valueOf())
console.log(new LosslessNumber('2.3e-500').valueOf())
Note that you can implement your own strategy for conversion by just getting the value as string via .toString()
, and using util functions like isInteger
, isSafeNumber
, getUnsafeNumberReason
, and toSafeNumberOrThrow
to convert it to a numeric value.
-
.toString() : string
Get the string representation of the lossless number.
Properties
{boolean} .isLosslessNumber : true
Lossless numbers contain a property isLosslessNumber
which can be used to
check whether some variable contains LosslessNumber.
Utility functions
-
isInteger(value: string) : boolean
Test whether a string contains an integer value, like '2300'
or 10
.
-
isNumber(value: string) : boolean
Test whether a string contains a numeric value, like '2.4'
or '1.4e+3'
.
-
isSafeNumber(value: string, config?: { approx: boolean }): boolean
Test whether a string contains a numeric value which can be safely represented by a JavaScript number
without losing any information. Returns false when digits would be truncated of an integer or decimal, or when the number would overflow or underflow. When passing { approx: true }
as config, the function will be less strict and allow losing insignificant digits of a decimal value. Examples:
isSafeNumber('1.55e3')
isSafeNumber('2e500')
isSafeNumber('2e-500')
isSafeNumber('9123372036854000123')
isSafeNumber('0.66666666666666666667')
isSafeNumber('9123372036854000123', { approx: true })
isSafeNumber('0.66666666666666666667', { approx: true })
-
toSafeNumberOrThrow(value: string, config?: { approx: boolean }) : number
Convert a string into a number when it is safe to do so, otherwise throw an informative error.
-
getUnsafeNumberReason(value): UnsafeNumberReason | undefined
When the provided value
is an unsafe number, describe what the reason is: overflow
, underflow
, truncate_integer
, truncate_float
. Returns undefined
when the value is safe.
-
isLosslessNumber(value: unknown) : boolean
Test whether a value is a LosslessNumber
.
-
toLosslessNumber(value: number) : LosslessNumber
Convert a number
into a LosslessNumber
. The function will throw an exception when the number
is exceeding the maximum safe limit of 15 digits (hence being truncated itself) or is NaN
or Infinity
.
-
parseLosslessNumber(value: string) : LosslessNumber
The default numberParser
used by parse
. Creates a LosslessNumber
from a string containing a numeric value.
-
parseNumberAndBigInt(value: string) : number | bigint
A custom numberParser
that can be used by parse
. The parser will convert integer values into bigint
, and converts al other values into a regular number
.
-
reviveDate(key, value)
Revive strings containing an ISO 8601 date string into a JavaScript Date
object. This reviver is not turned on by default because there is a small risk of parsing a text field that accidentally contains a date into a Date
. Whether reviveDate
is safe to use depends on the use case. Usage:
import { parse, reviveDate } from '@diotoborg/quos-accusantium'
const data = parse('["2022-08-25T09:39:19.288Z"]', reviveDate)
An alternative solution is to stringify a Date
in a specific recognizable object like {'$date':'2022-08-25T09:39:19.288Z'}
, and use a reviver and replacer to turn this object into a Date
and vice versa.
Alternatives
Similar libraries:
Test
To test the library, first install dependencies once:
npm install
To run the unit tests:
npm test
To build the library and run the unit tests and integration tests:
npm run build-and-test
Lint
Run linting:
npm run lint
Fix linting issues automatically:
npm run format
Benchmark
To run a benchmark to compare the performance with the native JSON
parser:
npm run benchmark
(Spoiler: @diotoborg/quos-accusantium
is much slower than native)
Build
To build a bundled and minified library (ES5), first install the dependencies once:
npm install
Then bundle the code:
npm run build
This will generate an ES module output and an UMD bundle in the folder ./.lib
which can be executed in browsers and node.js and used in the browser.
Release
To release a new version:
$ npm run release
This will:
- lint
- test
- build
- increment the version number
- push the changes to git, add a git version tag
- publish the npm package
To try the build and see the change list without actually publishing:
$ npm run release-dry-run
License
Released under the MIT license.