Security News
GitHub Removes Malicious Pull Requests Targeting Open Source Repositories
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.
lossless-json
Advanced tools
The lossless-json npm package allows for the parsing and stringifying of JSON data without losing precision for numbers. This is particularly useful for applications that require high precision in numerical data, such as financial calculations or scientific computations.
Parse JSON without losing precision
This feature allows you to parse JSON strings containing large numbers without losing precision. The parsed numbers are represented as LosslessNumber objects.
{"const losslessJSON = require('lossless-json');":"","const jsonString = '{\"bigNumber\": 123456789012345678901234567890}';":"","const parsed = losslessJSON.parse(jsonString);":"","console.log(parsed.bigNumber.toString());":"// '123456789012345678901234567890'"}
Stringify JSON with high precision numbers
This feature allows you to stringify objects containing LosslessNumber instances, ensuring that the precision of large numbers is maintained in the resulting JSON string.
{"const losslessJSON = require('lossless-json');":"","const obj = { bigNumber: losslessJSON.LosslessNumber('123456789012345678901234567890') };":"","const jsonString = losslessJSON.stringify(obj);":"","console.log(jsonString);":"// '{\"bigNumber\":123456789012345678901234567890}'"}
Custom reviver and replacer functions
This feature allows you to use custom reviver and replacer functions during parsing and stringifying, providing more control over how data is processed.
{"const losslessJSON = require('lossless-json');":"","const jsonString = '{\"bigNumber\": 123456789012345678901234567890}';":"","const parsed = losslessJSON.parse(jsonString, (key, value) => {":""," if (typeof value === 'string' && !isNaN(value)) {":""," return losslessJSON.LosslessNumber(value);":""," }":""," return value;":"","});":"","console.log(parsed.bigNumber.toString());":"// '123456789012345678901234567890'"}
The bignumber.js package provides arbitrary-precision arithmetic for JavaScript. While it does not handle JSON parsing and stringifying directly, it can be used in conjunction with JSON.parse and JSON.stringify to manage large numbers. Unlike lossless-json, it requires manual handling of big numbers.
The json-bigint package is designed to handle JSON with large integers. It uses BigInt to parse and stringify JSON, ensuring that large numbers are not lost. However, it does not support custom reviver and replacer functions like lossless-json.
Parse JSON without risk of losing numeric information.
import { parse, stringify } from 'lossless-json'
const text = '{"decimal":2.370,"long":9123372036854000123,"big":2.3e+500}'
// JSON.parse will lose some digits and a whole number:
console.log(JSON.stringify(JSON.parse(text)))
// '{"decimal":2.37,"long":9123372036854000000,"big":null}'
// WHOOPS!!!
// LosslessJSON.parse will preserve all numbers and even the formatting:
console.log(stringify(parse(text)))
// '{"decimal":2.370,"long":9123372036854000123,"big":2.3e+500}'
The following in-depth article explains what happens there: Why does JSON.parse corrupt large numbers and how to solve this?
How does it work? The library works exactly the same as the native JSON.parse
and JSON.stringify
. The difference is that lossless-json
preserves information of big numbers. lossless-json
parses numeric values not as a regular number but as a LosslessNumber
, a lightweight class which stores the numeric value as a string. One can perform regular operations with a LosslessNumber
, and it will throw an error when this would result in losing information.
When to use? If you have to deal with JSON data that contains long
values for example, coming from an application like C++, Java, or C#. The trade-off is that lossless-json
is slower than the native JSON.parse
and JSON.stringify
functions, so be careful when performance is a bottleneck for you.
Features:
bigint
.Date
(turned off by default).BigNumber
, bigint
, number
, or a mix of them.JSON.parse
and JSON.stringify
.Install via npm:
npm install lossless-json
Parsing and stringification works as you're used to:
import { parse, stringify } from 'lossless-json'
const json = parse('{"foo":"bar"}') // {foo: 'bar'}
const text = stringify(json) // '{"foo":"bar"}'
Numbers are parsed into a LosslessNumber
, which can be used like a regular number in numeric operations. Converting to a number will throw an error when this would result in losing information due to truncation, overflow, or underflow.
import { parse } from 'lossless-json'
const text = '{"normal":2.3,"long":123456789012345678901,"big":2.3e+500}'
const json = parse(text)
console.log(json.normal.isLosslessNumber) // true
console.log(json.normal.valueOf()) // number, 2.3
// LosslessNumbers can be used as regular numbers
console.log(json.normal + 2) // number, 4.3
// but the following operation will throw an error as it would result in information loss
console.log(json.long + 1)
// throws Error: Cannot safely convert LosslessNumber to number:
// "123456789012345678901" will be parsed as 123456789012345680000 and lose information
JavaScript natively supports bigint
: big integers that can hold a large number of digits, instead of the about 15 digits that a regular number
can hold. It is a typical use case to want to parse integer numbers into a bigint
, and all other values into a regular number
. This can be achieved with a custom numberParser
:
import { parse, isInteger } from 'lossless-json'
// parse integer values into a bigint, and use a regular number otherwise
export function customNumberParser(value) {
return isInteger(value) ? BigInt(value) : parseFloat(value)
}
const text = '[123456789123456789123456789, 2.3, 123]'
const json = parse(text, null, customNumberParser)
// output:
// [
// 123456789123456789123456789n, // bigint
// 2.3, // number
// 123n // bigint
// ]
You can adjust the logic to your liking, using utility functions like isInteger
, isNumber
, isSafeNumber
. The number parser shown above is included in the library and is named parseNumberAndBigInt
.
If you want parse a json string into an object with regular numbers, but want to validate that no numeric information is lost, you write your own number parser and use isSafeNumber
to validate the numbers:
import { parse, isSafeNumber } from 'lossless-json'
function parseAndValidateNumber(value) {
if (!isSafeNumber(value)) {
throw new Error(`Cannot safely convert value '${value}' into a number`)
}
return parseFloat(value)
}
// will parse with success if all values can be represented with a number
let json = parse('[1,2,3]', undefined, parseAndValidateNumber)
console.log(json) // [1, 2, 3] (regular numbers)
// will throw an error when some of the values are too large to represent correctly as number
try {
let json = parse('[1,2e+500,3]', undefined, parseAndValidateNumber)
} catch (err) {
console.log(err) // throws Error 'Cannot safely convert value '2e+500' into a number'
}
To use the library in conjunction with your favorite BigNumber library, for example decimal.js. You have to define a custom number parser and stringifier:
import { parse, stringify } from 'lossless-json'
import Decimal from 'decimal.js'
const parseDecimal = (value) => new Decimal(value)
const decimalStringifier = {
test: (value) => Decimal.isDecimal(value),
stringify: (value) => value.toString()
}
// parse JSON, operate on a Decimal value, then stringify again
const text = '{"value":2.3e500}'
const json = parse(text, undefined, parseDecimal) // {value: new Decimal('2.3e500')}
const output = {
// {result: new Decimal('4.6e500')}
result: json.value.times(2)
}
const str = stringify(output, undefined, undefined, [decimalStringifier])
// '{"result":4.6e500}'
The library is compatible with the native JSON.parse
and JSON.stringify
, and also comes with the optional reviver
and replacer
arguments that allow you to serialize for example data classes in a custom way. Here is an example demonstrating how you can stringify a Date
in a different way than the built-in reviveDate
utility function.
The following example stringifies a Date
as an object with a $date
key instead of a string, so it is uniquely recognizable when parsing the structure:
import { parse, stringify } from 'lossless-json'
// stringify a Date as a unique object with a key '$date', so it is recognizable
function customDateReplacer(key, value) {
if (value instanceof Date) {
return {
$date: value.toISOString()
}
}
return value
}
function isJSONDateObject(value) {
return value && typeof value === 'object' && typeof value.$date === 'string'
}
function customDateReviver(key, value) {
if (isJSONDateObject(value)) {
return new Date(value.$date)
}
return value
}
const record = {
message: 'Hello World',
timestamp: new Date('2022-08-30T09:00:00Z')
}
const text = stringify(record, customDateReplacer)
console.log(text)
// output:
// '{"message":"Hello World","timestamp":{"$date":"2022-08-30T09:00:00.000Z"}}'
const parsed = parse(text, customDateReviver)
console.log(parsed)
// output:
// {
// action: 'create',
// timestamp: new Date('2022-08-30T09:00:00.000Z')
// }
The LosslessJSON.parse()
function parses a string as JSON, optionally transforming the value produced by parsing.
{string} text
The string to parse as JSON. See the JSON object for a description of JSON syntax.{(key: string, value: unknown) => unknown} [reviver]
If a function, prescribes how the value originally produced by parsing is transformed, before being returned.{function(value: string) : unknown} [parseNumber]
Pass an optional custom number parser. Input is a string, and the output can be any numeric value: number
, bigint
, LosslessNumber
, or a custom BigNumber
library. By default, all numeric values are parsed into a LosslessNumber
.{unknown}
Returns the Object corresponding to the given JSON text.The LosslessJSON.stringify()
function converts a JavaScript value to a JSON string, optionally replacing values if a replacer function is specified, or optionally including only the specified properties if a replacer array is specified.
{unknown} value
The value to convert to a JSON string.{((key: string, value: unknown) => unknown) | Array.<string | number>} [replacer]
A function that alters the behavior of the stringification process, or an array with strings or numbers that serve as a whitelist for selecting the properties of the value object to be included in the JSON string. If this value is null
or not provided, all properties of the object are included in the resulting JSON string.{number | string | undefined} [space]
A string
or number
that is used to insert white space into the output JSON string for readability purposes. If this is a number
, it indicates the number of space characters to use as white space. Values less than 1 indicate that no space should be used. If this is a string
, the string
is used as white space. If this parameter is not provided (or is null
), no white space is used.{Array<{test: (value: unknown) => boolean, stringify: (value: unknown) => string}>} [numberStringifiers]
An optional list with additional number stringifiers, for example to serialize a BigNumber
. The output of the function must be valid stringified JSON number. When undefined
is returned, the property will be deleted from the object. The difference with using a replacer
is that the output of a replacer
must be JSON and will be stringified afterwards, whereas the output of the numberStringifiers
is already stringified JSON.{string | undefined}
Returns the string representation of the JSON object.numberStringifiers
does not return valid output.new LosslessNumber(value: number | string) : LosslessNumber
.valueOf(): number | bigint
Convert the LosslessNumber
into a regular number
or bigint
. A number
is returned for safe numbers and decimal values that only lose some insignificant digits. A bigint
is returned for large integer numbers. An Error
is thrown for values that will overflow or underflow. Examples:
// a safe number
console.log(new LosslessNumber('23.4').valueOf())
// number 23.4
// a decimal losing insignificant digits
console.log(new LosslessNumber('0.66666666666666666666667').valueOf())
// number 0.6666666666666666
// a large integer
console.log(new LosslessNumber('9123372036854000123').valueOf())
// bigint 9123372036854000123
// a value that will overflow
console.log(new LosslessNumber('2.3e+500').valueOf())
// Error: Cannot safely convert to number: the value '2.3e+500' would overflow and become Infinity
// a value that will underflow
console.log(new LosslessNumber('2.3e-500').valueOf())
// Error: Cannot safely convert to number: the value '2.3e-500' would underflow and become 0
Note that you can implement your own strategy for conversion by just getting the value as string via .toString()
, and using util functions like isInteger
, isSafeNumber
, getUnsafeNumberReason
, and toSafeNumberOrThrow
to convert it to a numeric value.
.toString() : string
Get the string representation of the lossless number.
{boolean} .isLosslessNumber : true
Lossless numbers contain a property isLosslessNumber
which can be used to
check whether some variable contains LosslessNumber.isInteger(value: string) : boolean
Test whether a string contains an integer value, like '2300'
or 10
.
isNumber(value: string) : boolean
Test whether a string contains a numeric value, like '2.4'
or '1.4e+3'
.
isSafeNumber(value: string, config?: { approx: boolean }): boolean
Test whether a string contains a numeric value which can be safely represented by a JavaScript number
without losing any information. Returns false when digits would be truncated of an integer or decimal, or when the number would overflow or underflow. When passing { approx: true }
as config, the function will be less strict and allow losing insignificant digits of a decimal value. Examples:
isSafeNumber('1.55e3') // true
isSafeNumber('2e500') // false
isSafeNumber('2e-500') // false
isSafeNumber('9123372036854000123') // false
isSafeNumber('0.66666666666666666667') // false
isSafeNumber('9123372036854000123', { approx: true }) // false
isSafeNumber('0.66666666666666666667', { approx: true }) // true
toSafeNumberOrThrow(value: string, config?: { approx: boolean }) : number
Convert a string into a number when it is safe to do so, otherwise throw an informative error.
getUnsafeNumberReason(value): UnsafeNumberReason | undefined
When the provided value
is an unsafe number, describe what the reason is: overflow
, underflow
, truncate_integer
, truncate_float
. Returns undefined
when the value is safe.
isLosslessNumber(value: unknown) : boolean
Test whether a value is a LosslessNumber
.
toLosslessNumber(value: number) : LosslessNumber
Convert a number
into a LosslessNumber
. The function will throw an exception when the number
is exceeding the maximum safe limit of 15 digits (hence being truncated itself) or is NaN
or Infinity
.
parseLosslessNumber(value: string) : LosslessNumber
The default numberParser
used by parse
. Creates a LosslessNumber
from a string containing a numeric value.
parseNumberAndBigInt(value: string) : number | bigint
A custom numberParser
that can be used by parse
. The parser will convert integer values into bigint
, and converts al other values into a regular number
.
reviveDate(key, value)
Revive strings containing an ISO 8601 date string into a JavaScript Date
object. This reviver is not turned on by default because there is a small risk of parsing a text field that accidentally contains a date into a Date
. Whether reviveDate
is safe to use depends on the use case. Usage:
import { parse, reviveDate } from 'lossless-json'
const data = parse('["2022-08-25T09:39:19.288Z"]', reviveDate)
// output:
// [
// new Date('2022-08-25T09:39:19.288Z')
// ]
An alternative solution is to stringify a Date
in a specific recognizable object like {'$date':'2022-08-25T09:39:19.288Z'}
, and use a reviver and replacer to turn this object into a Date
and vice versa.
Similar libraries:
To test the library, first install dependencies once:
npm install
To run the unit tests:
npm test
To build the library and run the unit tests and integration tests:
npm run build-and-test
Run linting:
npm run lint
Fix linting issues automatically:
npm run format
To run a benchmark to compare the performance with the native JSON
parser:
npm run benchmark
(Spoiler: lossless-json
is much slower than native)
To build a bundled and minified library (ES5), first install the dependencies once:
npm install
Then bundle the code:
npm run build
This will generate an ES module output and an UMD bundle in the folder ./.lib
which can be executed in browsers and node.js and used in the browser.
To release a new version:
$ npm run release
This will:
To try the build and see the change list without actually publishing:
$ npm run release-dry-run
Released under the MIT license.
FAQs
Parse JSON without risk of losing numeric information
The npm package lossless-json receives a total of 117,156 weekly downloads. As such, lossless-json popularity was classified as popular.
We found that lossless-json demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.
Security News
RubyGems.org has added a new "maintainer" role that allows for publishing new versions of gems. This new permission type is aimed at improving security for gem owners and the service overall.
Security News
Node.js will be enforcing stricter semver-major PR policies a month before major releases to enhance stability and ensure reliable release candidates.