Schema validation with static type inference
Features
- deriving artifacts from a
Schema
:
- guards
- assertions
- decoders
- encoders
- fast-check arbitraries
- pretty printing
- custom artifact compilers
- custom
Schema
combinators
Introduction
Welcome to the documentation for @fp-ts/schema
, a library for defining and using schemas to validate and transform data in TypeScript. @fp-ts/schema
allows you to define a Schema
that describes the structure and data types of a piece of data, and then use that Schema
to perform various operations such as decoding from unknown
, encoding to unknown
, verifying that a value conforms to a given Schema
. @fp-ts/schema
also provides a number of other features, including the ability to derive various artifacts such as Arbitrary
s, JSONSchema
s, and Pretty
s from a Schema
, as well as the ability to customize the library through the use of custom artifact compilers and custom Schema
combinators.
Requirements
- TypeScript 4.7 or newer
- The
strict
flag enabled in your tsconfig.json
file - The
exactOptionalPropertyTypes
flag enabled in your tsconfig.json
file
Getting started
To get started with @fp-ts/schema
, you will need to install the library using npm or yarn:
npm install @fp-ts/schema
yarn add @fp-ts/schema
Once you have installed the library, you can import the necessary types and functions from the @fp-ts/schema/Schema
module.
import * as S from "@fp-ts/schema/Schema";
Defining a schema
To define a Schema
, you can use the provided struct
function to define a new Schema
that describes an object with a fixed set of properties. Each property of the object is described by a Schema
, which specifies the data type and validation rules for that property.
For example, consider the following Schema
that describes a person object with a name
property of type string
and an age
property of type number
:
import * as S from "@fp-ts/schema/Schema";
const Person = S.struct({
name: S.string,
age: S.number,
});
You can also use the union
function to define a Schema
that describes a value that can be one of a fixed set of types. For example, the following Schema
describes a value that can be either a string
or a number
:
const StringOrNumber = S.union(S.string, S.number);
In addition to the provided struct
and union
functions, @fp-ts/schema
also provides a number of other functions for defining Schema
s, including functions for defining arrays, tuples, and records.
Once you have defined a Schema
, you can use the Infer
type to extract the inferred type of the data described by the Schema
.
For example, given the Person
Schema
defined above, you can extract the inferred type of a Person
object as follows:
interface Person extends S.Infer<typeof Person> {}
Decoding
To use the Schema
defined above to decode a value from unknown
, you can use the decode
function from the @fp-ts/schema/Parser
module:
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
const Person = S.struct({
name: S.string,
age: S.number,
});
const decodePerson = P.decode(Person);
const result1 = decodePerson({ name: "Alice", age: 30 });
if (PE.isSuccess(result1)) {
console.log(result1.right);
}
const result2 = decodePerson(null);
if (PE.isFailure(result2)) {
console.log(result2.left);
}
The decodePerson
function returns a value of type ParseResult<A>
, which is a type alias for Either<NonEmptyReadonlyArray<ParseError>, A>
, where NonEmptyReadonlyArray<ParseError>
represents a list of errors that occurred during the decoding process and A
is the inferred type of the data described by the Schema
. A successful decode will result in a Right
, containing the decoded data. A Right
value indicates that the decode was successful and no errors occurred. In the case of a failed decode, the result will be a Left
value containing a list of ParseError
s.
The decodeOrThrow
function is used to decode a value and throw an error if the decoding fails.
It is useful when you want to ensure that the value being decoded is in the correct format, and want to throw an error if it is not.
try {
const person = P.decodeOrThrow(Person)({});
console.log(person);
} catch (e) {
console.error("Decoding failed:");
console.error(e);
}
Excess properties
When using a Schema
to decode a value, any properties that are not specified in the Schema
will result in a decoding error. This is because the Schema
is expecting a specific shape for the decoded value, and any excess properties do not conform to that shape.
However, you can use the isUnexpectedAllowed
option to allow excess properties while decoding. This can be useful in cases where you want to be permissive in the shape of the decoded value, but still want to catch any potential errors or unexpected values.
Here's an example of how you might use isUnexpectedAllowed
:
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
const Person = S.struct({
name: S.string,
age: S.number,
});
console.log(
"%o",
P.decode(Person)(
{
name: "Bob",
age: 40,
email: "bob@example.com",
},
{ isUnexpectedAllowed: true }
)
);
All errors
The allErrors
option is a feature that allows you to receive all decoding errors when attempting to decode a value using a schema. By default only the first error is returned, but by setting the allErrors
option to true
, you can receive all errors that occurred during the decoding process. This can be useful for debugging or for providing more comprehensive error messages to the user.
Here's an example of how you might use allErrors
:
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
const Person = S.struct({
name: S.string,
age: S.number,
});
console.log(
"%o",
P.decode(Person)(
{
name: "Bob",
age: "abc",
email: "bob@example.com",
},
{ allErrors: true }
)
);
Encoding
To use the Schema
defined above to encode a value to unknown
, you can use the encode
function:
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
import { pipe } from "@fp-ts/data/Function";
import { parseNumber } from "@fp-ts/schema/data/parser";
const Age = pipe(S.string, parseNumber);
const Person = S.struct({
name: S.string,
age: Age,
});
const encoded = P.encode(Person)({ name: "Alice", age: 30 });
if (PE.isSuccess(encoded)) {
console.log(encoded.right);
}
Note that during encoding, the number value 30
was converted to a string "30"
.
Formatting errors
To format errors when a decode
or an encode
function fails, you can use the format
function from the @fp-ts/schema/formatter/Tree
module.
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
import { format } from "@fp-ts/schema/formatter/Tree";
const Person = S.struct({
name: S.string,
age: S.number,
});
const result = P.decode(Person)({});
if (PE.isFailure(result)) {
console.error("Decoding failed:");
console.error(format(result.left));
}
Assertions
The is
function provided by the @fp-ts/schema/Parser
module represents a way of verifying that a value conforms to a given Schema
. is
is a refinement that takes a value of type unknown
as an argument and returns a boolean
indicating whether or not the value conforms to the Schema
.
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
const Person = S.struct({
name: S.string,
age: S.number,
});
const isPerson = P.is(Person);
console.log(isPerson({ name: "Alice", age: 30 }));
console.log(isPerson(null));
console.log(isPerson({}));
The asserts
function takes a Schema
and an optional error message as arguments, and returns a function that takes an input value and checks if it matches the schema. If it does not match the schema, it throws an error with the specified message.
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
const Person = S.struct({
name: S.string,
age: S.number,
});
const assertsPerson: P.InferAsserts<typeof Person> = P.asserts(Person);
try {
assertsPerson({ name: "Alice", age: "30" });
} catch (e) {
console.error("The input does not match the schema:");
console.error(e);
}
assertsPerson({ name: "Alice", age: 30 });
The arbitrary
function provided by the @fp-ts/schema/Arbitrary
module represents a way of generating random values that conform to a given Schema
. This can be useful for testing purposes, as it allows you to generate random test data that is guaranteed to be valid according to the Schema
.
import * as S from "@fp-ts/schema/Schema";
import * as A from "@fp-ts/schema/Arbitrary";
import * as fc from "fast-check";
const Person = S.struct({
name: S.string,
age: S.number,
});
const PersonArbitrary = A.arbitrary(Person)(fc);
console.log(fc.sample(PersonArbitrary, 2));
Pretty print
The pretty
function provided by the @fp-ts/schema/Pretty
module represents a way of pretty-printing values that conform to a given Schema
.
You can use the pretty
function to create a human-readable string representation of a value that conforms to a Schema
. This can be useful for debugging or logging purposes, as it allows you to easily inspect the structure and data types of the value.
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Pretty";
const Person = S.struct({
name: S.string,
age: S.number,
});
const PersonPretty = P.pretty(Person);
console.log(PersonPretty({ name: "Alice", age: 30 }));
Understanding Schemas
A schema is a description of a data structure that can be used to generate various artifacts from a single declaration.
From a technical point of view a schema is just a typed wrapper of an AST
value:
interface Schema<in out A> {
readonly ast: AST;
}
The AST
type represents a tiny portion of the TypeScript AST, roughly speaking the part describing ADTs (algebraic data types),
i.e. products (like structs and tuples) and unions.
This means that you can define your own schema constructors / combinators as long as you are able to manipulate the AST
value accordingly, let's see an example.
Say we want to define a pair
schema constructor, which takes a Schema<A>
as input and returns a Schema<readonly [A, A]>
as output.
First of all we need to define the signature of pair
import * as S from "@fp-ts/schema/Schema";
declare const pair: <A>(schema: S.Schema<A>) => S.Schema<readonly [A, A]>;
Then we can implement the body using the APIs exported by the @fp-ts/schema/AST
module:
import * as S from "@fp-ts/schema/Schema";
import * as AST from "@fp-ts/schema/AST";
import * as O from "@fp-ts/data/Option";
const pair = <A>(schema: S.Schema<A>): S.Schema<readonly [A, A]> => {
const element = AST.element(
schema.ast,
false
);
const tuple = AST.tuple(
[element, element],
O.none,
true
);
return S.make(tuple);
};
This example demonstrates the use of the low-level APIs of the AST
module, however, the same result can be achieved more easily and conveniently by using the high-level APIs provided by the Schema
module.
const pair = <A>(schema: S.Schema<A>): S.Schema<readonly [A, A]> =>
S.tuple(schema, schema);
Please note that the S.tuple
API is a convenient utility provided by the library, but it can also be easily defined and implemented in userland.
export const tuple = <Elements extends ReadonlyArray<Schema<any>>>(
...elements: Elements
): Schema<{ readonly [K in keyof Elements]: Infer<Elements[K]> }> =>
makeSchema(
AST.tuple(
elements.map((schema) => AST.element(schema.ast, false)),
O.none,
true
)
);
Annotations
One of the fundamental requirements in the design of @fp-ts/schema
is that it is extensible and customizable. Customizations are achieved through "annotations". Each node contained in the AST of @fp-ts/schema/AST
contains an annotations: Record<string | symbol, unknown>
field that can be used to attach additional information to the schema.
Let's see some examples:
import { pipe } from "@fp-ts/data/Function";
import * as S from "@fp-ts/schema/Schema";
const Password = pipe(
S.string,
S.message(() => "not a string"),
S.nonEmpty,
S.message(() => "required"),
S.maxLength(10),
S.message((s) => `${s} is too long`),
S.identifier("Password"),
S.title("password"),
S.description(
"A password is a string of characters used to verify the identity of a user during the authentication process"
),
S.examples(["1Ki77y", "jelly22fi$h"]),
S.documentation(`
jsDoc documentation...
`)
);
The example shows some built-in combinators to add meta information, but users can easily add their own meta information by defining a custom combinator.
Here's an example of how to add a deprecated
annotation:
import * as S from "@fp-ts/schema/Schema";
import * as AST from "@fp-ts/schema/AST";
import { pipe } from "@fp-ts/data/Function";
const DeprecatedId = "some/unique/identifier/for/the/custom/annotation";
const deprecated = <A>(self: S.Schema<A>): S.Schema<A> =>
S.make(AST.annotation(self.ast, DeprecatedId, true));
const schema = pipe(S.string, deprecated);
console.log(schema);
Annotations can be read using the getAnnotation
helper, here's an example:
import * as O from "@fp-ts/data/Option";
const isDeprecated = <A>(schema: S.Schema<A>): boolean =>
pipe(
AST.getAnnotation<boolean>(DeprecatedId)(schema.ast),
O.getOrElse(() => false)
);
console.log(isDeprecated(S.string));
console.log(isDeprecated(schema));
Basic usage
Primitives
import * as S from "@fp-ts/schema/Schema";
S.string;
S.number;
S.bigint;
S.boolean;
S.symbol;
S.object;
S.json;
S.undefined;
S.void;
S.any;
S.unknown;
S.never;
Literals
S.null;
S.literal("a");
S.literal("a", "b", "c");
S.literal(1);
S.literal(2n);
S.literal(true);
Template literals
The templateLiteral
combinator allows you to create a schema for a TypeScript template literal type.
S.templateLiteral(S.literal("a"), S.string);
const EmailLocaleIDs = S.literal("welcome_email", "email_heading");
const FooterLocaleIDs = S.literal("footer_title", "footer_sendoff");
S.templateLiteral(S.union(EmailLocaleIDs, FooterLocaleIDs), S.literal("_id"));
Filters
Note. Please note that the use of filters do not alter the type of the Schema
. They only serve to add additional constraints to the decoding process.
Strings
pipe(S.string, S.maxLength(5));
pipe(S.string, S.minLength(5));
pipe(S.string, nonEmpty());
pipe(S.string, S.length(5));
pipe(S.string, S.pattern(regex));
pipe(S.string, S.startsWith(string));
pipe(S.string, S.endsWith(string));
pipe(S.string, S.includes(searchString));
Numbers
pipe(S.number, S.greaterThan(5));
pipe(S.number, S.greaterThanOrEqualTo(5));
pipe(S.number, S.lessThan(5));
pipe(S.number, S.lessThanOrEqualTo(5));
pipe(S.number, S.int());
pipe(S.number, S.nonNaN());
pipe(S.number, S.finite());
Native enums
enum Fruits {
Apple,
Banana,
}
S.enums(Fruits);
Nullables
S.nullable(S.string);
Unions
S.union(S.string, S.number);
Tuples
S.tuple(S.string, S.number);
Append a required element
pipe(S.tuple(S.string, S.number), S.element(S.boolean));
Append an optional element
pipe(S.tuple(S.string, S.number), S.optionalElement(S.boolean));
Append a rest element
pipe(S.tuple(S.string, S.number), S.rest(S.boolean));
Arrays
S.array(S.number);
Non empty arrays
S.nonEmptyArray(S.number);
Structs
S.struct({ a: S.string, b: S.number });
Optional fields
S.struct({ a: S.string, b: S.number, c: S.optional(S.boolean) });
Pick
pipe(S.struct({ a: S.string, b: S.number }), C.pick("a"));
Omit
pipe(S.struct({ a: S.string, b: S.number }), C.omit("a"));
Partial
S.partial(S.struct({ a: S.string, b: S.number }));
Records
String keys
S.record(S.string, S.string);
S.record(S.union(S.literal("a"), S.literal("b")), S.string);
Keys refinements
S.record(pipe(S.string, S.minLength(2)), S.string);
Symbol keys
S.record(C.symbol, S.string);
Template literal keys
S.record(S.templateLiteral(S.literal("a"), S.string), S.string);
Extend
The extend
combinator allows you to add additional fields or index signatures to an existing Schema
or Schema
.
pipe(
S.struct({ a: S.string, b: S.string }),
S.extend(S.struct({ c: S.boolean })),
S.extend(S.record(S.string, S.string))
);
InstanceOf
In the following section, we demonstrate how to use the instanceOf
combinator to create a Schema
for a class instance.
class Test {
constructor(readonly name: string) {}
}
pipe(S.object, S.instanceOf(Test));
Recursive types
The lazy
combinator is useful when you need to define a Schema
that depends on itself, like in the case of recursive data structures. In this example, the Category
schema depends on itself because it has a field subcategories
that is an array of Category
objects.
interface Category {
readonly name: string;
readonly subcategories: ReadonlyArray<Category>;
}
const Category: S.Schema<Category> = S.lazy(() =>
S.struct({
name: S.string,
subcategories: S.array(Category),
})
);
Here's an example of two mutually recursive schemas, Expression
and Operation
, that represent a simple arithmetic expression tree.
interface Expression {
readonly type: "expression";
readonly value: number | Operation;
}
interface Operation {
readonly type: "operation";
readonly operator: "+" | "-";
readonly left: Expression;
readonly right: Expression;
}
const Expression: S.Schema<Expression> = S.lazy(() =>
S.struct({
type: S.literal("expression"),
value: S.union(S.number, Operation),
})
);
const Operation: S.Schema<Operation> = S.lazy(() =>
S.struct({
type: S.literal("operation"),
operator: S.union(S.literal("+"), S.literal("-")),
left: Expression,
right: Expression,
})
);
Transformations
In some cases, we may need to transform the output of a schema to a different type. For instance, we may want to parse a string into a number, or we may want to transform a date string into a Date
object.
To perform these kinds of transformations, the @fp-ts/schema
library provides the transform
and transformOrFail
combinators.
The transform
combinator takes a target schema, a transformation function from the source type to the target type, and a reverse transformation function from the target type back to the source type. It returns a new schema that applies the transformation function to the output of the original schema before returning it. If the original schema fails to decode a value, the transformed schema will also fail.
import * as S from "@fp-ts/schema/Schema";
const stringSchema: S.Schema<string> = S.string;
const tupleSchema: S.Schema<[string]> = S.tuple(S.string);
const decode = (s: string): [string] => [s];
const encode = ([s]: [string]): string => s;
const transformedSchema: S.Schema<[string]> = pipe(
stringSchema,
S.transform(tupleSchema, decode, encode)
);
In the example above, we defined a schema for the string
type and a schema for the tuple type [string]
. We also defined the functions decode
and encode
that convert a string
into a tuple and a tuple into a string
, respectively. Then, we used the transform
combinator to convert the string schema into a schema for the tuple type [string]
. The resulting schema can be used to decode values of type string
into values of type [string]
.
The transformOrFail
combinator works in a similar way, but allows the transformation function to return a ParseResult
object, which can either be a success or a failure. This allows us to specify custom error messages in case the transformation fails.
Here's an example of the transformOrFail
combinator which converts a string
into a boolean
:
import { pipe } from "@fp-ts/data/Function";
import * as PE from "@fp-ts/schema/ParseError";
import type { ParseResult } from "@fp-ts/schema/Parser";
import * as S from "@fp-ts/schema/Schema";
const stringSchema: S.Schema<string> = S.string;
const booleanSchema: S.Schema<boolean> = S.boolean;
const decode = (s: string): ParseResult<boolean> =>
s === "true"
? PE.success(true)
: s === "false"
? PE.success(false)
: PE.failure(PE.transform("string", "boolean", s));
const encode = (b: boolean): ParseResult<string> => PE.success(String(b));
const transformedSchema: S.Schema<boolean> = pipe(
stringSchema,
S.transformOrFail(booleanSchema, decode, encode)
);
parseNumber
In the following section, we demonstrate how to use the parseNumber
combinator to convert a string
schema to a number
schema and parse string inputs into numbers. The parseNumber
combinator allows parsing special values such as NaN
, Infinity
, and -Infinity
in addition to regular numbers.
import * as S from "@fp-ts/schema/Schema";
import { parseNumber } from "@fp-ts/schema/data/parser";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
const schema = parseNumber(S.string);
const decode = P.decode(schema);
expect(decode("1")).toEqual(PE.success(1));
expect(decode("-1")).toEqual(PE.success(-1));
expect(decode("1.5")).toEqual(PE.success(1.5));
expect(decode("NaN")).toEqual(PE.success(NaN));
expect(decode("Infinity")).toEqual(PE.success(Infinity));
expect(decode("-Infinity")).toEqual(PE.success(-Infinity));
console.error(decode("a"));
Option
The option
combinator allows you to specify that a field in a schema may be either an optional value or null
. This is useful when working with JSON data that may contain null
values for optional fields.
In the example below, we define a schema for an object with a required a
field of type string
and an optional b
field of type number
. We use the option
combinator to specify that the b
field may be either a number
or null
.
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
import * as O from "@fp-ts/data/Option";
const schema = S.struct({
a: S.string,
b: S.option(S.number),
});
const decode = P.decode(schema);
expect(decode({ a: "hello", b: 1 })).toEqual(
PE.success({ a: "hello", b: O.some(1) })
);
expect(decode({ a: "hello", b: null })).toEqual(
PE.success({ a: "hello", b: O.none })
);
ReadonlySet
In the following section, we demonstrate how to use the fromValues
combinator to decode a ReadonlySet
from an array of values.
import * as S from "@fp-ts/schema/Schema";
import { fromValues } from "@fp-ts/schema/data/ReadonlySet";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
const schema = fromValues(S.number);
const decode = P.decode(schema);
expect(decode([1, 2, 3])).toEqual(PE.success(new Set([1, 2, 3])));
ReadonlyMap
In the following section, we demonstrate how to use the fromEntries
combinator to decode a ReadonlyMap
from an array of entries.
import * as S from "@fp-ts/schema/Schema";
import { fromEntries } from "@fp-ts/schema/data/ReadonlyMap";
import * as P from "@fp-ts/schema/Parser";
import * as PE from "@fp-ts/schema/ParseError";
const schema = fromEntries(S.number, S.string);
const decode = P.decode(schema);
expect(
decode([
[1, "a"],
[2, "b"],
[3, "c"],
])
).toEqual(
PE.success(
new Map([
[1, "a"],
[2, "b"],
[3, "c"],
])
)
);
Adding new data types
The easiest way to define a new data type is through the filter
combinator.
import * as S from "@fp-ts/schema/Schema";
import * as P from "@fp-ts/schema/Parser";
const LongString = pipe(
S.string,
S.filter((s) => s.length >= 10, {
message: () => "a string at least 10 characters long",
})
);
console.log(P.decodeOrThrow(LongString)("a"));
It is good practice to add as much metadata as possible so that it can be used later by introspecting the schema.
const LongString = pipe(
S.string,
S.filter((s) => s.length >= 10, {
message: () => "a string at least 10 characters long",
identifier: "LongString",
jsonSchema: { minLength: 10 },
description:
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua",
})
);
Documentation
License
The MIT License (MIT)