github.com/prataprc/goparsec
Package parsec provides a library of parser-combinators. The basic idea behind parsec module is that, it allows programmers to compose basic set of terminal parsers, a.k.a tokenizers and compose them together as a tree of parsers, using combinators like: And, OrdChoice, Kleene, Many, Maybe. To begin with there are four basic Types that needs to be kept in mind while creating and composing parsers, Scanner, an interface type that encapsulates the input text. A built in scanner called SimpleScanner is supplied along with this package. Developers can also implement their own scanner types. Following example create a new instance of SimpleScanner, using an input text: Nodify, callback function is supplied while combining parser functions. If the underlying parsing logic matches with i/p text, then callback will be dispatched with list of matching ParsecNode. Value returned by callback function will further be used as ParsecNode item in higher-level list of ParsecNodes. Parser, simple parsers are functions that matches i/p text for specific patterns. Simple parsers can be combined using one of the supplied combinators to construct a higher level parser. A parser function takes a Scanner object and applies the underlying parsing logic, if underlying logic succeeds Nodify callback is dispatched and a ParsecNode and a new Scanner object (with its cursor moved forward) is returned. If parser fails to match, it shall return the input scanner object as it is, along with nil ParsecNode. ParsecNode, an interface type encapsulates one or more tokens from i/p text, as terminal node or non-terminal node. If input text is going to be a single token like `10` or `true` or `"some string"`, then all we need is a single Parser function that can tokenize the i/p text into a terminal node. But our applications are seldom that simple. Almost all the time we need to parse the i/p text for more than one tokens and most of the time we need to compose them into a tree of terminal and non-terminal nodes. This is where combinators are useful. Package provides a set of combinators to help combine terminal parsers into higher level parsers. They are, All the above mentioned combinators accept one or more parser function as arguments, either by value or by reference. The reason for allowing parser argument by reference is to be able to define recursive parsing logic, like parsing nested arrays: Parsers for standard set of tokens are supplied along with this package. Most of these parsers return Terminal type as ParseNode. All of the terminal parsers, except End and NoEnd return Terminal type as ParsecNode. While End and NoEnd return a boolean type as ParsecNode. This is an experimental feature to use CSS like selectors for quering an Abstract Syntax Tree (AST). Types, APIs and methods associated with AST and Queryable are unstable, and are expected to change in future. While Scanner, Parser, ParsecNode types are re-used in AST and Queryable, combinator functions are re-implemented as AST methods. Similarly type ASTNodify is to be used instead of Nodify type. Otherwise all the parsec techniques mentioned above are equally applicable on AST. Additionally, following points are worth noting while using AST,
Readme
A library to construct top-down recursive backtracking parsers using parser-combinators. Before proceeding you might want to take at peep at theory of parser combinators. As for this package, it provides:
To construct syntax-trees based on detailed grammar try with AST struct
NOTE that AST object is a recent development and expect user to adapt to newer versions
Every combinator should confirm to the following signature,
// ParsecNode type defines a node in the AST
type ParsecNode interface{}
// Parser function parses input text, higher order parsers are
// constructed using combinators.
type Parser func(Scanner) (ParsecNode, Scanner)
// Nodify callback function to construct custom ParsecNode.
type Nodify func([]ParsecNode) ParsecNode
Combinators take a variable number of parser functions and return a new parser function.
Builtin scanner library manages the input buffer and implements a cursor into the buffer. Create a new scanner instance,
s := parsec.NewScanner(text)
The scanner library supplies method like Match(pattern),
SkipAny(pattern) and Endof(), refer to for
more information on each of these methods.
Panics are to be expected when APIs are misused. Programmers might choose to ignore errors, but not panics. For example:
Clone the repository run the benchmark suite
$ cd expr/
$ go test -test.bench=. -test.benchmem=true
$ cd json/
$ go test -test.bench=. -test.benchmem=true
To run the example program,
# to parse expression
$ go run tools/parsec/parsec.go -expr "10 + 29"
# to parse JSON string
$ go run tools/parsec/parsec.go -json '{ "key1" : [10, "hello", true, null, false] }'
If your project is using goparsec you can raise an issue to list them under this section.
Maybe combinator
master is the development branch.FAQs
Package parsec provides a library of parser-combinators. The basic idea behind parsec module is that, it allows programmers to compose basic set of terminal parsers, a.k.a tokenizers and compose them together as a tree of parsers, using combinators like: And, OrdChoice, Kleene, Many, Maybe. To begin with there are four basic Types that needs to be kept in mind while creating and composing parsers, Scanner, an interface type that encapsulates the input text. A built in scanner called SimpleScanner is supplied along with this package. Developers can also implement their own scanner types. Following example create a new instance of SimpleScanner, using an input text: Nodify, callback function is supplied while combining parser functions. If the underlying parsing logic matches with i/p text, then callback will be dispatched with list of matching ParsecNode. Value returned by callback function will further be used as ParsecNode item in higher-level list of ParsecNodes. Parser, simple parsers are functions that matches i/p text for specific patterns. Simple parsers can be combined using one of the supplied combinators to construct a higher level parser. A parser function takes a Scanner object and applies the underlying parsing logic, if underlying logic succeeds Nodify callback is dispatched and a ParsecNode and a new Scanner object (with its cursor moved forward) is returned. If parser fails to match, it shall return the input scanner object as it is, along with nil ParsecNode. ParsecNode, an interface type encapsulates one or more tokens from i/p text, as terminal node or non-terminal node. If input text is going to be a single token like `10` or `true` or `"some string"`, then all we need is a single Parser function that can tokenize the i/p text into a terminal node. But our applications are seldom that simple. Almost all the time we need to parse the i/p text for more than one tokens and most of the time we need to compose them into a tree of terminal and non-terminal nodes. This is where combinators are useful. Package provides a set of combinators to help combine terminal parsers into higher level parsers. They are, All the above mentioned combinators accept one or more parser function as arguments, either by value or by reference. The reason for allowing parser argument by reference is to be able to define recursive parsing logic, like parsing nested arrays: Parsers for standard set of tokens are supplied along with this package. Most of these parsers return Terminal type as ParseNode. All of the terminal parsers, except End and NoEnd return Terminal type as ParsecNode. While End and NoEnd return a boolean type as ParsecNode. This is an experimental feature to use CSS like selectors for quering an Abstract Syntax Tree (AST). Types, APIs and methods associated with AST and Queryable are unstable, and are expected to change in future. While Scanner, Parser, ParsecNode types are re-used in AST and Queryable, combinator functions are re-implemented as AST methods. Similarly type ASTNodify is to be used instead of Nodify type. Otherwise all the parsec techniques mentioned above are equally applicable on AST. Additionally, following points are worth noting while using AST,
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
A hospital in Mobile, Alabama, agreed to a settlement in a landmark ransomware death lawsuit, but is now reportedly reconsidering the agreement and refusing to pay.
Security News
A new report explores how advancements in LLMs are enhancing cyber threats, including polymorphic malware, personalized spearphishing, and the risk of hijacking customer service bots.
Security News
ESLint has approved an RFC that adds support for TypeScript configuration files, which is aimed at improving the developer experience and recognizing changes in the evolving JavaScript ecosystem.