relude-parse

relude-parse

Overview

ReludeParse is a pure-functional string parsing library for ReasonML/Bucklescript, built on top of the functional programming types and type classes provided by Relude.

ReludeParse was influenced by purescript-string-parsers, parsihax, and atto, and several of the parsers in the Haskell ecosystem.

Getting started

Install the library

// Install the library from npm
npm install --save-dev relude relude-parse
// add relude and relude-parse to your bsconfig.json

Import the types, functions, operators, etc.

ReludeParse is the root namespace module.

In ReasonML/Bucklescript, you can get access to a module's functions in a variety of ways, here are a few examples:

// Method 1: Global open (not recommended)
open ReludeParse.Parser;
let _ = anyDigit <* eof |> runParser("1");

// Method 2: Local open (good but more verbose)
let _ = ReludeParse.Parser.(anyDigit <* eof |> runParser("1"));

// Method 3: Module alias with small global open (tradeoff between 1 and 2 above)
module P = ReludeParse.Parser;
open P.Infix; // Get all the infix operators in scope
let _ = P.anyDigit <* P.eof |> P.runParser("1");

I recommend using method 3 for most cases, but if you are running into conflicts, use method 2. If you just want to live your life, use method 1.

For examples below, I'm going to assume method 3.

As a side note, the relude ecosystem prefers the |> pipe operator over ->, so most functions are "significant data last" style, and the "significant data" is typically a value of type Parser.t('a). This allows you to do things like:

P.anyAlpha |> P.sepBy(str(","))

which is the same as below:

P.sepBy(str(","), P.anyAlpha)

Luckily, either way is perfectly legit, so use whichever style you prefer.

Parsers

ReludeParse.Parser.t('a) is a data type which encapsulates the ability to parse a string into a value of type 'a, or fail with a fixed ParseError.t type, which provides an error message and parse position.

ReludeParse provides a wide variety of low-level, medium-level, and high-level parsers which can be composed together to parse just about anything. Note that there are a few functions (like many) that are not stack safe, so beware when parsing very large strings of repeating characters. Someday these functions maybe become stack safe.

The parse errors are currently pretty minimal only giving an error message and parse position. Someday this may improve.

Run a parser

A parser will attempt to consume input from a string until it can successfully finish and produce a value or fail. The parser will only consume enough of the string to satisfy its own parsing needs (whatever it needs to produce the final value), and will leave the rest of the string for the next parser (if any).

To run a parser, simply pipe (or pass) it into the P.runParser function, along with the input string:

Success example:

// anyDigit will attempt to consume a single character, and succeeds if that character
// is a digit 0-9

// Pipe operator |>

P.anyDigit |> P.runParser("1") // Belt.Result.Ok("1")

// Or normal function application

P.runParser("1", P.anyDigit)

Failure example:

P.anyDigit |> P.runParser("!") // Belt.Result.Error(ParseError("Expected a digit"))

In the event of a failed parse, parsers will (in most cases) back-track so that an alternative parser (if supplied) can pick-up where the previous failed.

There is also a P.unParser function, which gives you access to more of the internals in the event of success or failure. It's called unParser because it unwraps or "lowers" a value of type Parser.t('a) to the raw parsing function contained within.

Mapping a function over a parser

A RelueParse.Parser is a functor, so we can map a pure function over the parser's value.

// Warning: int_of_string is unsafe (can throw) - this is just an example
P.anyDigit |> P.map(int_of_string) |> P.runParser("1") // Belt.Result.Ok(1)

// <$> map operator version.  <$> is traditionally "function on the left"
int_of_string <$> P.anyDigit |> P.runParser("1");

// <#> flipped map operator version - <#> is function on the right hand side,
// which might be more readable for some.
// When you see `<#>` just think `.map(...)` from JavaScript
P.anyDigit <#> int_of_string |> P.runParser("1");

Note: see the monad section for an information on how to handle or produce errors while parsing.

Parser.t('a) comes with all the bells and whistles that are granted to all functors in relude. This includes:

• the map function
• the <$> (map) operator
• flipMap
• the <#> (flipMap) operator
• the <$ operator
• the $> operator

These things are all provided "for free" via the Relude Functor Extensions

Note: Parser.t('a) is not a bi-functor - the error type is fixed to a known data type for simplicity.

Combining parsers (via Applicative)

A ReludeParse.Parser.t('a) is an applicative functor, so we can combine multiple parsers together using a variety of techniques.

// Combine two parsers into a tuple of the results (assuming all succeed)
P.tuple2(P.anyDigit, P.anyDigit) |> P.runParser("12") // Belt.Result.Ok(("1", "2"))

// <^> operator (operator version of tuple2)
P.anyDigit <^> P.anyDigit |> P.runParser("12") // Belt.Result.Ok(("1", "2"))

// Combine more parsers using tuple3 up to tuple5
P.tuple3(P.anyDigit, P.anyDigit, P.anyDigit)
|> P.runParser("123") // Belt.Result.Ok(("1", "2", "3"))

// Combine parse results using a function via map2 through map5
P.map2((a, b) => a + b, P.anyDigitAsInt, P.anyDigitAsInt)
|> P.runParser("12") // Belt.Result.Ok(3)

// Combine results from a tuple of parsers using mapTuple2 through mapTuple5
(P.anyDigitAsInt, P.anyDigitAsInt)
|> P.mapTuple2((a, b) => a + b)
|> P.runParser("12"); // Belt.Result.Ok(3)

// Use the *> operator to run two parsers, and only keep the result from the right side
// This is useful if you don't care what the left side parser produces (e.g. whitespace)
// but you need to consume that input.
// `ws` consumes all the whitespace it encounters and throws it away
P.ws
*> P.anyDigit
|> P.runParser("   3") // Belt.Result.Ok("3")

// Use the <* operator to run two parsers, and only keep the result from the left side
// This is useful if you don't care what the right side parser produces (e.g. whitespace)
// but you want to consume that input.
// It's common to use this operator with `eof` to make sure you've hit the end of the input
// (but you don't care about the value produced by `eof`).
// E.g. use both *> and <* to trim whitespace surrounding a value
P.ws
*> P.anyDigit
<* P.ws
<* P.eof
|> P.runParser("   3  ") // Belt.Result.Ok("3")

// ADVANCED: Incrementally collect parse results using a function and chained <$> map and <*> apply
// operators.
let add3 = (a, b, c) => a + b + c;
add3
<$> P.anyDigitAsInt
<*> P.anyDigitAsInt
<*> P.anyDigitAsInt
|> P.runParser("123"); // Belt.Result.Ok(6)

Many of these functions and operators come for free for any Applicative via Relude Apply Extensions

Sequencing parsers (via Monads)

A Parser.t('a) is also a monad, so you can put pure values directly into a parser using pure, and more importantly, you can sequence parsers using flatMap, bind, or the >>= operator. bind, flatMap and >>= all basically do the same thing - the take a Parser.t('a), a function from 'a => Parser.t('b) and give you a Parser.t('b). What this basically means is that you can "run" a parser to produce a value (note: I don't mean return a value, but produce a value inside your monadic flow), then use that value to create a new parser with which to continue processing. Note that you can't do that with the functors and applicative-based parsing, because map/apply/<*> and friends don't give you the opportunity to produce a new parser based on the value - you can only apply functions inside the context of a parser.

For intuition, flatMap is an apt name for this function because if you have a myToOfA: t('a), a function aToTOfB: 'a => t('b), and you do myTOfA |> map(aToTOfB) you'll get a t(t('b)). A monad has the ability to "flatten" itself when in a nested structure like this, i.e. flatten: t(t('b)) => t('b). In some languages flatten is often named join - like you are flattening or joining a nested structure into a single structure. flatMap can be implemented in terms of map and flatten, and flatten can be implmeented in terms of flatMap.

So flatMap is mapping a monadic function 'a => t('b) over a t('a), and then flattening the resulting t(t('b)) to just t('b).

The true power of monads is the ability to produce a new monadic value (i.e. a new parser) mid-flow, which can be used for producing and handling errors, or forking the parse flow to do something different, based on what you've previously parsed.

Note that monads have "fail-fast" semantics, because if a parser fails to produce a value, it's not possible for the next parser to accept a value (because there is none). In other words, if a parser fails at some point in a chain, the rest of the parsers will not run.

// Lift a pure value into a parser.
// As you can see the parser just produces the given value regardless of the string.
P.pure(3)
|> runParser("abcdef") // Belt.Result.Ok(3)

// Sequence parse operations using flatMap.
// In this example we read a single digit as an int, then use that value
// to read a series of letters, and expect to consume the whole input.
// This is sequencing because we use the result of one parser to determine
// the next parser to run.
P.anyDigitAsInt
|> P.flatMap(count => P.anyAlpha |> P.times(count) <* P.eof)
|> P.map(chars => Relude.List.String.join(chars))
|> P.runParser("3abc"); // Belt.Result.Ok("abc")

// Sequence using >>= (flatMap/bind) and <#> (map) operators.
// If you are coming from JS -
// Don't be afraid of the operators - when you see >>= read ".flatMap(...)"
// and when you see "<#>" read ".map(...)".  Eventually these will become
// second nature.
P.anyDigitAsInt
>>= (count => P.times(count, P.anyAlpha) <* P.eof)
<#> Relude.List.String.join
|> P.runParser("3abc"); // Belt.Result.Ok("abc")

Many of these functions come for free for any Monad via Relude Monad Extensions

Add validation and error handling in a parse chain

You can also use the monadic behavior to optionally fail a parse inside a bind/flatMap/>>= function. Note that you can't fail a parse inside a map because map uses a pure function from 'a => 'b, so there's no way to indicate failure of the parse - you are only allowed to produce a new value 'b inside the context of an existing parser.

P.anyDigitAsInt
>>= (
  count =>
    if (count >= 5) {
      // P.fail is a parser that always fails with the given message
      // just like P.pure always succeeds with the given value.
      // Using >>= and fail is a common way to inject validations and raise errors.
      P.fail("The count cannot be >= 5");
    } else {
      // Now that we have a valid count, carry on
      P.times(count, P.anyAlpha) <* eof;
    }
)
<#> Relude.List.String.join
|> runParser("9abc") // Belt.Result.Error(ParseError("The count cannot be >= 5"))

The filter function in ReludeParse is basically for this purpose. Filter produces its own generic error message if the predicate fails, but you can customize it like below:

P.anyDigitAsInt
|> P.filter(a => a > 5)
<?> "Expected an int greater than 5";

Trying multiple parsers (via Alt)

ReludeParse.Parser.t('a) is also an alt functor, which means you can try one parser, and if it fails, try another, as many times as you want.

The <|> operator is used for this - think of the <|> operator as an orElse function.

P.anyDigit <|> P.anyAlpha |> P.runParser("9") // Belt.Result.Ok("9")
P.anyDigit <|> P.anyAlpha |> P.runParser("a") // Belt.Result.Ok("a")
P.anyDigit <|> P.anyAlpha |> P.runParser("!") // Belt.Result.Error(...)

<|> can be chained as many times as you want - it attempts each parser left-to-right.

P.str("a") <|> P.str("b") <|> P.str("c") <|> P.str("d") ...and so on

If none of the parsers succeed, it will return the error of the last parser, so a common technique is to use <?> to add a custom error message at the end

P.str("a") <|> P.str("b") <|> P.str("c") <?> "Expected a, b, or c"

Sometimes when using <|> with more complex parsers, the first parser might consume some input before failing, which might mess up the next parser in the <|> chain. Use the tries function to force a parser to back-track all the way to it's original position if it fails.

// Without tries, this fails, because the first parser consumes the 9, then fails,
// but the next parser wants to consume a digit then a letter.  Using tries makes the
// parser fully back-track on failure if it had consumed any input.
P.tries(P.anyDigit *> P.anyDigit) // parse a digit, throw it away, then parse another digit
<|> (P.anyDigit *> P.anyAlpha) // parse a digit,throw it away, then parse a letter
|> P.runParser("9a") // Belt.Result.Ok("a")

Customizing the error message

Use the <?> operator to put a custom error message on a parser. This is useful if you are composing a more complex parser from smaller parsers, and want a more meaningful error message if the parser fails.

P.many1(P.anyDigit)
<?> "Expected one or more digits"
|> P.runParser("abc") // Belt.Result.Error(ParseError("Expected one or more digits"))

Checking that all input is consumed

To make sure that all the input in the string has been consumed, use the eof (end-of-file) parser. It's common to use <* eof to parse the end of input, because <* will just keep what's on the left side of eof.

P.anyDigit <* P.eof // Succeeds for "3" but fails for "3 "

Debugging

Use tap to inspect the result of a successful parse, and the parse position.

Use the tapLog function to inject some basic logging anywhere in a parser composition.

anyDigit *> anyDigit *> anyAlpha |> tapLog // etc.

Examples

IPv4

E.g. 127.0.0.1

There are many different ways to compose a parser to parse values like this. Below are just some examples to show different techniques.

type t = | IPv4(int, int, int, int);
let make = (a, b, c, d) => IPv4(a, b, c, d);

// Using a tuple and mapTuple4
// parse a short (up to 255) and the dot separators
(
  anyPositiveShort <* str("."),
  anyPositiveShort <* str("."),
  anyPositiveShort <* str("."),
  anyPositiveShort
)
|> mapTuple4(make)
|> runParser("127.0.0.1");

// Using nested flatMaps and a final map at the end.
// These are nested because we have to collect each value as we go, and it has to
// be in scope at the end when we want to construct our final value.
// Note: language support for sequences of monadic binds (e.g. do notation or the
// upcoming let+/let* bindings in OCaml, this becomes a beautiful, flat expression,
// almost like imperative code, but with pure FP data structures and functions!
anyPositiveShort
>>= (
  a =>
    str(".")
    >>= (
      _ =>
        anyPositiveShort
        >>= (
          b =>
            str(".")
            >>= (
              _ =>
                anyPositiveShort
                >>= (
                  c =>
                    str(".")
                    >>= (
                      _ =>
                        anyPositiveShort
                        <#> (
                          d =>
                            make(a, b, c, d)
                        )
                    )
                )
            )
        )
    )
)
|> runParser("127.0.0.1");

// With a monadic flow, another technique is to map each successive result into
// an ever-expanding tuple, and pass the values along that way. This allows you
// to have a more flat structure, at the
// expense of wrapping and unwrapping tuples at each step.

// Using <$> and <*> (with <* and *> helpers)
// Our make function is (int, int, int, int) => IPv4
// The first map <$> creates a `Parser.t((int, int, int) => IPv4)`
// and each successive <*> fills another slot in our function,
// until we finally collect the 4 args.
// The `<* str(".")` reads a ".", but throw it away.
make 
<$> anyPositiveShort // collect a positive short
<* str(".")          // read and ignore .
<*> anyPositiveShort // collect a positive short
<* str(".")          // read and ignore .
<*> anyPositiveShort // collect a positive short
<* str(".")          // read and ignore .
<*> anyPositiveShort // collect a positive short
|> runParser("127.0.0.1")

// Using sepBy
// Note that sepBy produces a list of the values produced by the value parser,
// so we have to manually validate that we got the correct number in our list.
// This is done using `>>=`, so we can fail the parse with the `fail` function,
// which produces a failing parser.  If we get the 4 values we need, we use pure
// to create a parser that produces our desired IPv4 value.
anyPositiveShort
|> sepBy(str("."))
>>= (
  // sepBy gives us a list, so we have to pick the parts out
  shorts =>
    switch (shorts) {
    | [a, b, c, d] =>
      // Use pure here because we need to wrap the result in a parser to satisfy
      // the type signature of the >>= function
      pure(make(a, b, c, d))
    | _ => fail("Expected exactly 4 shorts separated by .")
           // fail produces a `Parser.t(_)` that will always fail
    }
)
|> runParser("127.0.0.1")

See the code and tests for more examples.

API Documentation

For more details, examples, tests, etc., please refer to the code. Below is a possibly incomplete list of parser functions that come with ReludeParse.

Basic operations

Function	Description	Example
runParser	runs a parser with an input string to produce a Belt.Result with either the value or a ParseError.t
unParser	runs a parser with an input string to produce a Belt.Result with either the value or a ParseError.t (with some additional metadata compared to runParser)

Core FP functions and operators

Function	Description	Example
map/<$>/<#>/<$/$>	functor functions for mapping pure functions over a parser
apply/<*>/<*/*>	applicative functions for combining parsers
<^>	combine two parsers to produce a tuple of results
tuple2-5	combine parsers to produce tuples of results
map2-5	combine parsers using a function to combine the results
mapTuple2-5	combine a tuple of parsers using a function to combine the results
pure	lift a pure value into a parser that always succeeds with the value
unit	lift a pure () value into a parser that always succeeds with ()
flatMap/bind/>>=	map a function over a parser that can produce a new parser - used for sequencing parsers

Parsers also get all of the additional utilities provided by all the Relude typeclass extensions.

Logging/side effects

Function	Description	Example
tap	runs a side effect function in a parse change, and forwards the given result along
tapLog	logs some information about the current parse position

Error customization/handling

Function	Description	Example
withError/flipWithError/<?>	provide a custom error message
throwError	Create a parser that fails with the given ParseError.t
fail	Create a parser that fails with the given string message
catchError	Handle a failed parse by converting a ParseError.t into a new parser

Repeated values

Function	Description	Example
many	Run any parser 0 or more times to produce a [] of values (like * in regex)
many1	Run any parser 1 or more times to produce a Relude.NonEmptyList (aka Nel) of values (like + in regex) - the result is a Nel because we are guaranteed to find at least one value, otherwise the parser will fail
times	run a parser count times and produce a [] of results
times2-5	run a parser exactly twice (up to 5) to produce a tuple of results
timesMin	run a parser at least n times to produce a list of results
timesMax	run a parser at most n times to produce a list of results
timesMinMax	run a parser at least n times and at most m times to produce a list of results
manyUntilWithEnd	parse 0 or more values until a terminator is reached, producing the results and the consumed terminator
many1UntilWithEnd	parse 1 or more values until a terminator is reached, producing the results and the consumed terminator
manyUntil	parse 0 or more values until a terminator is reached, producing the results and discarding the consumed terminator
many1Until	parse 1 or more values until a terminator is reached, producing the results and discarding the consumed terminator
manyUntilPeekWithEnd	parse 0 or more values until a terminator is reached, producing the results and the terminator, without consuming the terminator
many1UntilPeekWithEnd	parse 1 or more values until a terminator is reached, producing the results and the terminator, without consuming the terminator
manyUntilPeek	parse 0 or more values until a terminator is reached, producing the results and discarding the terminator, without consuming the terminator
many1UntilPeek	parse 1 or more values until a terminator is reached, producing the results and discarding the terminator, without consuming the terminator

Optional/default values

Function	Description	Example
opt	attempt a parser, and wrap a success in Some and convert a failure to None
orDefault	attempt a parser, and if it fails, produce a default value
orUnit	attempt a parser, and if it fails, produce unit

Delimited values

Function	Description	Example
between	parse a value inside opening and closing delimiters	(abc)
sepBy	parse zero or more values separated by a delimiter	a,b,c,d
sepBy1	parse one or more values separated by a delimiter	a,b,c,d
sepByOptEnd	parse zero or more values separated by a delimiter, optionally ending with the delimiter	a,b,c,d or a,b,c,d,
sepByOptEnd1	parse one or more values separated by a delimiter, optionally ending with the delimiter	a,b,c,d or a,b,c,d,
sepByWithEnd	parse zero or more delimited values ending with the delimiter	a,b,c,
sepByWithEnd1	parse one or more delimited values ending with the delimiter	a,b,c,

Associative operators

Function	Description	Example
chainr1	parse values separated by a right-associative operator (useful for parsing math expressions)
chainl1	parse values separated by a left-associative operator (useful for parsing math expressions)

Trying different parsers

Function	Description	Example
tries	Tries a parser, and backtracks all the way to the original start position for the parser on failure. This is useful if you are using a more complex parser that might consume some input successfully before it fails. The default behavior is to only backtrack to the start of the failure, whereas this function forces the parse to backtrack to the beginning of the complex parser's input.
alt/altLazy/orElse/orElseLazy/<pipe>	try a parser, and if it fails, try the other
anyOf	Attempts to parse a value using a list of potential parsers, tried from left to right

Validation/extraction

Function	Description	Example
lookAhead/peek	Run a parser to produce a value, without consuming any input
lookAheadNot/peekNot	Run a parser which fails if it produces a value, without consuming any input
filter	apply a predicate to the result of a parser to either continue or fail the parse
getSome	converts a parser of option('a) into a parser of 'a, failing if the value is None
getNonEmptyStr	converts a parser of string into a parser of a non-empty string, failing if the value is ""
getFst	converts a parser of ('a, 'b) into a parser of 'a
getSnd	converts a parser of ('a, 'b) into a parser of 'b

Text parsers

Function	Description	Example
eof	verify that the end of the input has been reached
orEOF	attempts a parser, and throws away the result, or if it fails, attempts the eof parser
anyChar	parses any single character
notChar	parses any single character except the given
anyStr	parses any string (WARNING: this will likely consume all remaining input)
anyNonEmptyStr	parses any non-empty ("") string
str	parses the given string
strIgnoreCase	parses the given string case-insensitively
anyCharBy	parses a character, and checks it with a predicate function
anyOfStr	parses any of the given strings
anyOfStrIgnoreCase	parses any of the given strings case insensitively
wsList	parses any amount of whitespace characters and returns them in a list of single chars
wsString	parses any amount of whitespace characters and returns them in a string
ws	parses any amount of whitespace characters throws them away (produces ())
anyCharNotIn	parses any single char not in the given list
anyCharNotInIgnoreCase	parses any single char not in the given list, case-insensitive
anyCharInRange	parses any character in the ASCII code range
anyNonDigit	parses any non-digit character
anyLowerCaseChar	parses a single lowercase letter char
anyUpperCaseChar	parses a single uppercase letter char
anyAlpha	parses any upper or lowercase letter char
anyAlphaOrDigit	parses any upper or lowercase letter or digit char
regex	parses a string matching the given regex
regexStr	parses a string matching the given regex string
leftParen	parses a (
rightParen	parses a )
betweenParens	parses a value inside ( and ), consuming extra whitespace padding
leftCurly	parses a {
rightCurly	parses a }
betweenCurlies	parses a value inside { and }, consuming extra whitespace padding
leftSquare	parses a [
rightSquare	parses a ]
betweenSquares	parses a value inside [ and ], consuming extra whitespace padding
leftAngle	parses a <
rightAngle	parses a >
betweenAngles	parses a value inside < and >, consuming extra whitespace padding
singleQuote	parses a '
betweenSingleQuotes	parses a value inside ' and '
doubleQuote	parses a '
betweenDoubleQuotes	parses a value inside " and "
backTick	parses a backtick
betweenBackTicks	parses a value inside backticks
cr	parses a \r
lf	parses a \n
crlf	parses a \r\n
eol	parses a \r\n, \r, or a \n
orEOL	runs a parser and discards the result, or if it fails, try the eol parser

Numeric parsers

Function	Description	Example
anyDigit	parses any single character, makes sure it's a digit 0-9 and produces it as a single character string
anyDigitAsInt	parses any single character, makes sure it's a digit 0-9 and converts it to an int
anyNonEmptyDigits	parses 1 or more consecutive digits as a string
anyNonZeroDigit	parses any non-zero digit character
anyNonZeroDigitAsInt	parses any non-zero digit character as an int
anyUnsignedInt	parses an integer with no + or - prefix
anyPositiveInt	parses a positive integer (optional + prefix)
anyNegativeInt	parses a negative integer (- prefix)
anyInt	parses any positive or negative int
anyUnsignedShort	parses a short int
anyDecimal	parses a decimal value with optional exponential notation
anyHexDigit	parses any hex digit 0-9 or a-f or A-F
anyNonZeroHexDigit	parses any hex digit 1-9 or a-f or A-F
anyBool	parses a bool true or false value

See the code and tests for a complete list of functions and examples.

Extra Utilities

ReludeParse comes with a few higher-level parsers for convenience and for educational purposes.

• ReludeParse.IPv4 - IPv4 addresses
• ReludeParse.IPv6 - IPv6 addresses
• ReludeParse.UUID - 8-4-4-4-12 UUIDs
• ReludeParse.NanpPhone - North American Numbering Plan (NANP) phone numbers

Other parsers and utilities for things like URLs and DateTimes can be found in other libraries like relude-url and relude-eon.