Serializr
Serialize and deserialize complex object graphs to JSON
Serializr is feature complete, and easily extendable. Since there are no active maintainers the project is frozen feature wise. Bug reports and well designed pull requests are welcome and will be addressed.
Want to maintain a small open source project or having great ideas for this project? We are looking for maintainers, so apply!
Introduction
Serializr is a utility library that helps converting json structures into complex object graphs and the other way around.
For a quick overview, read the introduction blog post
Features:
- (De)serialize objects created with a constructor / class
- (De)serialize primitive values
- (De)serialize nested objects, maps and arrays
- Resolve references asynchronously (during deserialization)
- Supports inheritance
- Works on any ES5 environment (if ES3 is needed file a feature request)
- Convenience decorators for ESNext / Typescript
- Ships with typescript / flow typings
- Generic solution that works well with for example MobX out of the box
Non-features:
- Serializr is not an ORM or data management library. It doesn't manage object instances, provided api's like fetch, search etc. If you are building such a thing though, serializr might definitely take care of the serialization part for you :-).
- Serializr is not a MobX specific (de)serialization mechanism, it is generic and should fit work with any type of model objects
Installation
From npm: npm install serializr --save
From CDN: https://unpkg.com/serializr which declares the global serializr
object.
Quick example:
import {
createModelSchema,
primitive,
reference,
list,
object,
identifier,
serialize,
deserialize
} from "serializr"
class User {
uuid = Math.floor(Math.random() * 10000)
displayName = "John Doe"
}
class Message {
message = "Test"
author = null
comments = []
}
function fetchUserSomewhere(uuid) {
const user = new User()
user.uuid = uuid
user.displayName = `John Doe ${uuid}`
return user
}
function findUserById(uuid, callback, context) {
callback(null, fetchUserSomewhere(uuid))
}
createModelSchema(Message, {
message: primitive(),
author: reference(User, findUserById),
comments: list(object(Message))
})
createModelSchema(User, {
uuid: identifier(),
displayName: primitive()
})
const message = deserialize(Message, {
message: "Hello world",
author: 17,
comments: [
{
message: "Welcome!",
author: 23
}
]
})
const json = serialize(message)
console.dir(message, { colors: true, depth: 10 })
Using decorators (optional)
With decorators (TypeScript or ESNext) building model schemas is even more trivial:
import {
createModelSchema,
primitive,
reference,
list,
object,
identifier,
serialize,
deserialize,
getDefaultModelSchema,
serializable
} from "serializr"
class User {
@serializable(identifier())
uuid = Math.random()
@serializable
displayName = "John Doe"
}
class Message {
@serializable
message = "Test"
@serializable(object(User))
author = null
@serializable(list(object(Message)))
comments = []
}
const message = deserialize(Message, {
message: "Hello world",
author: { uuid: 1, displayName: "Alice" },
comments: [
{
message: "Welcome!",
author: { uuid: 1, displayName: "Bob" }
}
]
})
console.dir(message, { colors: true, depth: 10 })
const json = serialize(message)
Decorator: Caveats
Babel 6.x does not allow decorators to self-reference during their creation, so the above code would not work for the Message class. Instead write:
class Message {
@serializable message = "Test"
@serializable(object(User))
author = null
comments = []
constructor() {
getDefaultModelSchema(Message).props["comments"] = list(object(Message))
}
}
Enabling decorators (optional)
TypeScript
Enable the compiler option experimentalDecorators
in tsconfig.json
or pass it as flag --experimentalDecorators
to the compiler.
Babel 7.x:
Install support for decorators: npm i --save-dev @babel/plugin-proposal-class-properties @babel/plugin-proposal-decorators
. And enable it in your .babelrc
file:
{
"presets": ["@babel/preset-env"],
"plugins": [
["@babel/plugin-proposal-decorators", { "legacy": true }],
["@babel/plugin-proposal-class-properties", { "loose": true }]
]
}
Babel 6.x:
Install support for decorators: npm i --save-dev babel-plugin-transform-decorators-legacy
. And enable it in your .babelrc
file:
{
"presets": ["es2015", "stage-1"],
"plugins": ["transform-decorators-legacy"]
}
Babel 5.x
{
"stage": 1
}
Probably you have more plugins and presets in your .babelrc
already, note that the order is important and transform-decorators-legacy
should come as first.
Concepts
The two most important functions exposed by serializr are serialize(modelschema?, object) -> json tree
and deserialize(modelschema, json tree) -> object graph
.
What are those model schemas?
ModelSchema
The driving concept behind (de)serialization is a ModelSchema.
It describes how model object instances can be (de)serialize to json.
A simple model schema looks like this:
const todoSchema = {
factory: context => new Todo(),
extends: ModelSchema,
props: {
modelfield: PropSchema
}
}
The factory
tells how to construct new instances during deserialization.
The optional extends
property denotes that this model schema inherits its props from another model schema.
The props section describes how individual model properties are to be (de)serialized. Their names match the model field names.
The combination fieldname: true
is simply a shorthand for fieldname: primitive()
For convenience, model schemas can be stored on the constructor function of a class.
This allows you to pass in a class reference wherever a model schema is required.
See the examples below.
PropSchema
PropSchemas contain the strategy on how individual fields should be serialized.
It denotes whether a field is a primitive, list, whether it needs to be aliased, refers to other model objects etc.
PropSchemas are composable. See the API section below for the details, but these are the built-in property schemas:
primitive()
: Serialize a field as primitive valueidentifier()
: Serialize a field as primitive value, use it as identifier when serializing references (see reference
)date()
: Serializes dates (as epoch number)alias(name, propSchema)
: Serializes a field under a different namelist(propSchema)
: Serializes an array based collectionmap(propSchema)
: Serializes an Map or string key based collectionmapAsArray(propSchema, keyPropertyName)
: Serializes a map to an array of elementsobject(modelSchema)
: Serializes an child model elementreference(modelSchema, lookupFunction?)
: Serializes a reference to another model elementcustom(serializeFunction, deserializeFunction)
: Create your own property serializer by providing two functions, one that converts modelValue to jsonValue, and one that does the inverse- There is a special prop schema:
"*": true
that serializes all enumerable, non mentioned values as primitive
It is possible to define your own prop schemas. You can define your own propSchema by creating a function that returns an object with the following signature:
{
serializer: (sourcePropertyValue: any) => jsonValue,
deserializer: (jsonValue: any, callback: (err, targetPropertyValue: any) => void, context?, currentPropertyValue?) => void
}
For inspiration, take a look at the source code of the existing ones on how they work, it is pretty straightforward.
Deserialization context
The context object is an advanced feature and can be used to obtain additional context-related information about the deserialization process.
context
is available as:
- first argument of factory functions
- third argument of the lookup callback of
ref
prop schema's (see below) - third argument of the
deserializer
of a custom propSchema
When deserializing a model elememt / property, the following fields are available on the context object:
json
: Returns the complete current json object that is being deserializedtarget
: The object currently being deserialized. This is the object that is returned from the factory function.parentContext
: Returns the parent context of the current context. For example if a child element is being deserialized, the context.target
refers to the current model object, and context.parentContext.target
refers to the parent model object that owns the current model object.args
: If custom arguments were passed to the deserialize
/ update
function, they are available as context.args
.
AdditionalPropArgs
A PropSchema can be further parameterized using AdditionalPropArgs. Currently, they can be used to specify lifecycle functions. During deserialization they can be useful, e.g. in case you want to
- react to errors in the deserialization on a value level and retry with corrected value,
- remove invalid items e.g. in arrays or maps,
- react to changes in field names, e.g. due to schema migration (i.e. only one-directional changes that cannot be dealt with by alias operators).
It is possible to define those functions by passing them as additional property arguments to the propSchema during its creation.
const myHandler = {
beforeDeserialize: function(
callback,
jsonValue,
jsonParentValue,
propNameOrIndex,
context,
propDef
) {
if (typeof jsonValue === "string") {
callback(null, jsonValue)
} else if (typeof jsonValue === "number") {
callback(null, jsonValue.toString())
} else {
callback(new Error("something went wrong before deserialization"))
}
},
afterDeserialize: function(
callback,
error,
newValue,
jsonValue,
jsonParentValue,
propNameOrIndex,
context,
propDef
) {
if (!error && newValue !== "needs change") {
callback(null, newValue)
} else if (!error && newValue === "needs change") {
callback(new Error(), "changed value")
} else {
callback(error)
}
}
}
class MyData {
@serializable(primitive(myHandler))
mySimpleField
}
A more detailed example can be found in test/typescript/ts.ts.
API
interface ModelSchema
<T>src
property targetClass
?: Clazz<any>
property factory
: (context: Context) => T
property props
: Props<T>
property extends
?: ModelSchema<any>
interface PropSchema
src
property pattern
?: undefined | { test: (propName: string) => boolean }
Filter properties to which this schema applies. Used with ModelSchema.props["*"]
.
property jsonname
?: undefined | string
property identifier
?: undefined | true
property paramNumber
?: undefined | number
type AdditionalPropArgs
= Pick<PropSchema, "beforeDeserialize"
| "afterDeserialize"
| "pattern"
> src
Can be passed to function which create PropSchema
s to set additional properties.
type AfterDeserializeFunc
= (callback: (err: any, value: any) => void, err: any, newValue: any, jsonValue: any, jsonParentValue: any, propNameOrIndex: string | number | symbol, context: Context, propDef: PropSchema) => void src
type BeforeDeserializeFunc
= (callback: (err: any, value: any) => void, jsonValue: any, jsonParentValue: any, propNameOrIndex: string | number, context: Context, propDef: PropSchema) => void src
type Clazz
<T> = { } src
type ClazzOrModelSchema
<T> = ModelSchema<T> | Clazz<T> src
type PropDef
= PropSchema | boolean | undefined src
type PropDeserializer
= (jsonValue: any, callback: (err?: any, targetPropertyValue?: any | typeof SKIP) => void, context: Context, currentPropertyValue?: any) => void src
type PropSerializer
= (sourcePropertyValue: any, key: string | number | symbol, sourceObject: any) => any | typeof SKIP src
type Props
<T> = { } src
true is shorthand for primitive().
false/undefined will be ignored
type RefLookupFunction
= (id: string, callback: (err: any, result: any) => void, context: Context) => void src
type RegisterFunction
= (id: any, object: any, context: Context) => void src
const SKIP
src
If you want to skip serialization or deserialization, you can use SKIP.
const schema = createSimpleSchema({
a: custom(
() => SKIP,
v => v,
),
})
serialize(s, { a: 4 })
deserialize(s, { "a": 4 })
class TodoState {
@serializable(list(object(Todo)))
@observable
todos: Todo[]
@serializable(
list(object(TodoCategory),
{ afterDeserialize: callback => callback(undefined, SKIP) }))
@computed
get categories() {
return this.todos.map(todo => todo.category)
}
}
function alias
(name: string, propSchema?: PropDef): PropSchema src
Alias indicates that this model property should be named differently in the generated json. Alias should be the outermost propschema.
createModelSchema(Todo, {
title: alias('task', primitive()),
})
serialize(new Todo('test'))
function cancelDeserialize
<T>(instance: T): void src
Cancels an asynchronous deserialization or update operation for the specified target object.
function createModelSchema
<T>(clazz: Clazz<T>, props: Props, factory?: undefined | ((context: Context) => T)): ModelSchema<T> src
Creates a model schema that (de)serializes an object created by a constructor function (class). The created model schema is associated by the targeted type as default model schema, see setDefaultModelSchema. Its factory method is () => new clazz()
(unless overriden, see third arg).
function Todo(title, done) {
this.title = title
this.done = done
}
createModelSchema(Todo, {
title: true,
done: true,
})
const json = serialize(new Todo('Test', false))
const todo = deserialize(Todo, json)
function createSimpleSchema
<T>(props: Props): ModelSchema<T> src
Creates a model schema that (de)serializes from / to plain javascript objects. Its factory method is: () => ({})
const todoSchema = createSimpleSchema({
title: true,
done: true,
})
const json = serialize(todoSchema, { title: 'Test', done: false })
const todo = deserialize(todoSchema, json)
function custom
(serializer: PropSerializer, deserializer: (jsonValue: any, context: Context, oldValue: any, callback: (err: any, result: any | typeof SKIP) => void) => void, additionalArgs?: AdditionalPropArgs): PropSchema src
Can be used to create simple custom propSchema. Multiple things can be done inside of a custom propSchema, like deserializing and serializing other (polymorphic) objects, skipping the serialization of something or checking the context of the obj being (de)serialized.
The custom
function takes two parameters, the serializer
function and the deserializer
function.
The serializer
function has the signature: (value, key, obj) => void
When serializing the object {a: 1}
the serializer
function will be called with serializer(1, 'a', {a: 1})
.
The deserializer
function has the following signature for synchronous processing (value, context, oldValue) => void
For asynchronous processing the function expects the following signature (value, context, oldValue, callback) => void
When deserializing the object {b: 2}
the deserializer
function will be called with deserializer(2, contextObj)
(contextObj reference).
const schemaDefault = createSimpleSchema({
a: custom(
v => v + 2,
v => v - 2
)
})
serialize(schemaDefault, { a: 4 })
deserialize(schemaDefault, { "a": 6 })
const schemaWithAsyncProps = createSimpleSchema({
a: custom(
v => v + 2,
(v, context, oldValue, callback) =>
somePromise(v, context, oldValue)
.then(result => callback(null, result - 2))
.catch(err => callback(err))
)
})
serialize(schemaWithAsyncProps, { a: 4 })
deserialize(schemaWithAsyncProps, { "a": 6 }, (err, res) => {
res
}
function custom
(serializer: PropSerializer, deserializer: (jsonValue: any, context: Context, oldValue: any) => any | typeof SKIP, additionalArgs?: AdditionalPropArgs): PropSchema src
Similar to primitive, serializes instances of Date objects
function deserialize
<T>(modelschema: ClazzOrModelSchema<T>, jsonArray: any[], callback?: undefined | ((err: any, result: T[]) => void), customArgs?: any): T[] src
Deserializes a json structure into an object graph.
This process might be asynchronous (for example if there are references with an asynchronous lookup function). The function returns an object (or array of objects), but the returned object might be incomplete until the callback has fired as well (which might happen immediately)
function deserialize
<T>(modelschema: ClazzOrModelSchema<T>, json: any, callback?: undefined | ((err: any, result: T) => void), customArgs?: any): T src
function getDefaultModelSchema
<T>(thing: any): ModelSchema<T> | undefined src
Returns the standard model schema associated with a class / constructor function
List indicates that this property contains a list of things. Accepts a sub model schema to serialize the contents
class SubTask {}
class Task {}
class Todo {}
createModelSchema(SubTask, {
title: true,
})
createModelSchema(Todo, {
title: true,
subTask: list(object(SubTask)),
})
const todo = deserialize(Todo, {
title: 'Task',
subTask: [
{
title: 'Sub task 1',
},
],
})
Similar to list, but map represents a string keyed dynamic collection. This can be both plain objects (default) or ES6 Map like structures. This will be inferred from the initial value of the targetted attribute.
Similar to map, mapAsArray can be used to serialize a map-like collection where the key is contained in the 'value object'. Example: consider Map<id: number, customer: Customer> where the Customer object has the id stored on itself. mapAsArray stores all values from the map into an array which is serialized. Deserialization returns a ES6 Map or plain object object where the keyPropertyName
of each object is used for keys. For ES6 maps this has the benefit of being allowed to have non-string keys in the map. The serialized json also may be slightly more compact.
object
indicates that this property contains an object that needs to be (de)serialized using its own model schema.
N.B. mind issues with circular dependencies when importing model schema's from other files! The module resolve algorithm might expose classes before createModelSchema
is executed for the target class.
class SubTask {}
class Todo {}
createModelSchema(SubTask, {
title: true,
})
createModelSchema(Todo, {
title: true,
subTask: object(SubTask),
})
const todo = deserialize(Todo, {
title: 'Task',
subTask: {
title: 'Sub task',
},
})
Optional indicates that this model property shouldn't be serialized if it isn't present.
Note that if we use optional
together with another prop schema such as custom
, the prop schema for custom
will be applied first and the result of that serialization will be used to feed into optional
.
createModelSchema(Todo, {
title: optional(primitive()),
user: optional(custom(value => value.name, () => SKIP))
})
serialize(new Todo())
Indicates that this field contains a primitive value (or Date) which should be serialized literally to json.
createModelSchema(Todo, {
title: primitive(),
})
serialize(new Todo('test'))
Indicates that this field is only need to putted in the serialized json or deserialized instance, without any transformations. Stay with its original value
createModelSchema(Model, {
rawData: raw(),
})
serialize(new Model({ rawData: { a: 1, b: [], c: {} } } }))
reference
can be used to (de)serialize references that point to other models.
The first parameter should be either a ModelSchema that has an identifier()
property (see identifier) or a string that represents which attribute in the target object represents the identifier of the object.
The second parameter is a lookup function that is invoked during deserialization to resolve an identifier to an object. Its signature should be as follows:
lookupFunction(identifier, callback, context)
where: 1. identifier
is the identifier being resolved 2. callback
is a node style calblack function to be invoked with the found object (as second arg) or an error (first arg) 3. context
see context.
The lookupFunction is optional. If it is not provided, it will try to find an object of the expected type and required identifier within the same JSON document
N.B. mind issues with circular dependencies when importing model schemas from other files! The module resolve algorithm might expose classes before createModelSchema
is executed for the target class.
class User {}
class Post {}
createModelSchema(User, {
uuid: identifier(),
displayname: primitive(),
})
createModelSchema(Post, {
author: reference(User, findUserById),
message: primitive(),
})
function findUserById(uuid, callback) {
fetch('http://host/user/' + uuid)
.then(userData => {
deserialize(User, userData, callback)
})
.catch(callback)
}
deserialize(
Post,
{
message: 'Hello World',
author: 234,
},
(err, post) => {
console.log(post)
}
)
function serializable
(propSchema: PropDef): (target: any, key: string, baseDescriptor?: PropertyDescriptor) => void src
Decorator that defines a new property mapping on the default model schema for the class it is used in.
When using typescript, the decorator can also be used on fields declared as constructor arguments (using the private
/ protected
/ public
keywords). The default factory will then invoke the constructor with the correct arguments as well.
class Todo {
@serializable(primitive())
title
@serializable
done
constructor(title, done) {
this.title = title
this.done = done
}
}
const json = serialize(new Todo('Test', false))
const todo = deserialize(Todo, json)
function serializable
(target: any, key: string, baseDescriptor?: PropertyDescriptor): void src
function serialize
<T>(modelSchema: ClazzOrModelSchema<T>, instance: T): any src
Serializes an object (graph) into json using the provided model schema. The model schema can be omitted if the object type has a default model schema associated with it. If a list of objects is provided, they should have an uniform type.
function serialize
<T>(instance: T): any src
function serializeAll
<T>(clazz: Clazz<T>): Clazz<T> src
The serializeAll
decorator can used on a class to signal that all primitive properties, or complex properties with a name matching a pattern
, should be serialized automatically.
@serializeAll
class Store {
a = 3
b
}
const store = new Store()
store.c = 5
store.d = {}
serialize(store)
class DataType {
@serializable
x
@serializable
y
}
@serializeAll(/^[a-z]$/, DataType)
class ComplexStore {
}
const store = new ComplexStore()
store.a = {x: 1, y: 2}
store.b = {}
store.somethingElse = 5
serialize(store)
function serializeAll
(pattern: RegExp, propertyType: PropDef | Clazz<any>): (clazz: Clazz<any>) => Clazz<any> src
function setDefaultModelSchema
<T>(clazz: Clazz<T>, modelSchema: ModelSchema<T>): ModelSchema<T> src
Sets the default model schema for class / constructor function. Everywhere where a model schema is required as argument, this class / constructor function can be passed in as well (for example when using object
or ref
.
When passing an instance of this class to serialize
, it is not required to pass the model schema as first argument anymore, because the default schema will be inferred from the instance type.
function update
(modelSchema: any, target: any, json: any, callback: any, customArgs?: any): void src
Recipes and examples
1. Plain schema with plain objects
const todoSchema = {
factory: () => {},
props: {
task: primitive(),
owner: reference("_userId", UserStore.findUserById),
subTasks: alias("children", list(object(todoSchema)))
}
}
const todo = deserialize(
todoSchema,
{ task: "grab coffee", owner: 17, children: [] },
(err, todo) => {
console.log("finished loading todos")
}
)
const todoJson = serialize(todoSchema, todo)
2. Create schema and store it on constructor
function Todo(parentTodo) {
this.parent = parentTodo
}
const todoSchema = {
factory: context => new Todo(context.parent),
props: {
task: primitive(),
owner: reference("_userId", UserStore.findUserById),
subTasks: alias("children", list(object(todoSchema)))
}
}
setDefaultModelSchema(Todo, todoSchema)
const todo = deserialize(
Todo,
{ task: "grab coffee", owner: 17, children: [] },
(err, todos) => {
console.log("finished loading todos")
}
)
const todoJson = serialize(todo)
3. Create schema for simple argumentless constructors
function Todo() {}
createModelSchema(Todo, {
task: primitive()
})
const todo = deserialize(
Todo,
{ task: "grab coffee", owner: 17, children: [] },
(err, todos) => console.log("finished loading todos")
)
const todoJson = serialize(todo)
4. Create schema for simple argumentless constructors using decorators
class Todo {
@serializable(primitive())
task = "Grab coffee"
@serializable(reference("_userId", UserStore.findUserById))
owner = null
@serializable(alias("children", list(object(todoSchema))))
subTasks = []
}
const todos = deserialize(
Todo,
[
{
task: "grab coffee",
owner: 17,
children: []
}
],
(err, todos) => console.log("finished loading todos")
)
const todoJson = serialize(todos)
5. use custom factory methods to reuse model object instances
const someTodoStoreById = {}
getDefaultModelSchema(Todo).factory = context => {
const json = context.json
if (someTodoStoreById[json.id]) return someTodoStoreById[json.id]
return (someTodoStoreById[json.id] = new Todo())
}
6. use custom arguments to inject stores to models
This pattern is useful to avoid singletons but allow to pass context specific data to constructors. This can be done by passing custom data to deserialize
/ update
as last argument,
which will be available as context.args
on all places where context is available:
class User {
constructor(someStore) {
}
}
createModelSchema(User, { username: true }, context => {
return new User(context.args.someStore)
})
const someStore = new SomeStore()
const user = deserialize(User, someJson, (err, user) => console.log("done"), {
someStore: someStore
})
7. Putting it together: MobX store with plain objects, classes and internal references
import { observable, computed } from "mobx"
import { serializable, identifier } from "serializr"
function randomId() {
return Math.floor(Math.random() * 100000)
}
export class Box {
@serializable(identifier())
id = randomId()
@serializable
@observable
x = 0
@serializable
@observable
y = 0
@serializable
@observable
location = 0
constructor(location, x, y) {
this.location = location
this.x = x
this.y = y
}
@serializable
@computed
get area() {
return this.x * this.y
}
}
export class Arrow {
@serializable(identifier())
id = randomId()
@serializable(reference(Box))
from
@serializable(reference(Box))
to
}
import { observable, transaction } from "mobx"
import { createSimpleSchema, identifier, list, serialize, deserialize, update } from "serializr"
import { Box, Arrow } from "./models"
const store = observable({
boxes: [],
arrows: [],
selection: null
})
const storeModel = createSimpleSchema({
boxes: list(object(Box)),
arrows: list(object(Arrow)),
selection: reference(Box)
})
store.boxes.push(new Box("Rotterdam", 100, 100), new Box("Vienna", 650, 300))
store.arrows.push({
id: randomId(),
from: store.boxes[0],
to: store.boxes[1]
})
function serializeState(store) {
return serialize(storeModel, store)
}
function deserializeState(store, json) {
transaction(() => {
update(storeModel, store, json)
})
}
console.dir(serializeState(store), { depth: 10, colors: true })
Future ideas