The Stardog Whisperer (error-tolerant parsers for languages used with Stardog)
Millan is the Stardog whisperer -- a set of parsers for languages used with Stardog (currently SPARQL, Turtle, TriG, Stardog Mapping Syntax 2 (SMS), Stardog Rules Syntax, the Turtle serialization of SHACL, and GraphQL).
Simply run:
npm install --save millan
You can then import Millan in Node.js or in the browser, using any preferred module format or bundler.
In Node/CommonJS:
const millan = require('millan');
// or, if you only need some part of millan (e.g., only _one_ of the parsers):
const graphql = require('millan/dist/standalone/millan.graphql');
As an ES module:
import * as millan from 'millan';
// or, if you only need some part of millan (e.g., only _one_ of the parsers):
import { StardogSparqlParser } from 'millan/dist/standalone/millan.sparql';
In the browser, importing everything at once:
<script src="path/to/millan/millan.js"></script>
In the browser, importing only what is needed:
<!-- Here, we first load any shared chunks needed by 'sms' (as indicated by the filename), then the SMS-related Javacript only -->
<script src="path/to/millan/dist/browser/millan.vendors~graphql~shacl~sms~sparql~srs~turtle.js"></script>
<script src="path/to/millan/dist/browser/millan.sms.js"></script>
When used in the browser via script tags, the API will be exposed on a global millan variable.
Import Millan or the specific parts of Millan that you need, e.g.:
import { W3SpecSparqlParser } from 'millan';
Parsing a document requires a parser instance, so you should get one:
const sparqlParser = new W3SpecSparqlParser();
Every parser instance has essentially the same API. They are all instances of the Parser base class from chevrotain, and they all implement the IStardogParser interface.
The two parser methods you'll use most often are parse and tokenize. Use
tokenize only when you want the parser instance to lex (tokenize) the
provided text:
sparqlParser.tokenize(`
PREFIX : <http://example.org/ns#>
SELECT *
WHERE { ?x ?y ?z }
`);
Result:
[
{
"image": "PREFIX",
"startOffset": 3,
"endOffset": 8,
"startLine": 2,
"endLine": 2,
"startColumn": 3,
"endColumn": 8,
"tokenTypeIdx": 43,
"tokenType": {
"PATTERN": {},
"tokenTypeIdx": 43,
"CATEGORIES": [],
"categoryMatches": [],
"categoryMatchesMap": {},
"tokenName": "PREFIX",
"isParent": false
}
},
/* . . . */
]
Use parse when you want the full parsing experience. Note that, because
Millan's parsers are error-tolerant, the parse method returns an object
that contains both a concrete syntax tree (CST)
and an array of any errors:
sparqlParser.parse(`
PREFIX : <http://example.org/ns#>
SELECT *
WHERE { ?x ?y ?z }
`);
Result:
{
"errors": [],
"cst": {
"name": "SparqlDoc",
"children": {
"Prologue": [
{
"name": "Prologue",
"children": {
"PrefixDecl": [
{
"name": "PrefixDecl",
"children": {
"PREFIX": [
{
"image": "PREFIX",
"startOffset": 3,
"endOffset": 8,
"startLine": 2,
"endLine": 2,
"startColumn": 3,
"endColumn": 8,
"tokenTypeIdx": 43,
"tokenType": {
"PATTERN": {},
"tokenTypeIdx": 43,
"CATEGORIES": [],
"categoryMatches": [],
"categoryMatchesMap": {},
"tokenName": "PREFIX",
"isParent": false
}
}
],
"PNAME_NS": [ /* . . . */ ],
/* . . . etc. . . . */
Helpfully, the name fields in the CSTs returned by Millan's parsers exactly
match, in nearly every case (we've removed some redundancy), the names of the
rules in the official EBNF for the corresponding grammar (e.g., SparqlDoc,
Prologue, PrefixDecl, etc.).
When there are errors in parsing, Millan's parsers still return a best-effort CST, as well as information including the spot where the error occurred, what the parser expected to find at that spot, what the parser actually found in that spot, the "context" (i.e., the position in the grammar's "rule stack"), and more:
sparqlParser.parse(`
PREFIX : <http://example.org/ns#>
SELECT 1
WHERE { ?x ?y ?z }
`);
Result:
{
"errors": [
{
"name": "NoViableAltException",
"message": "Expecting: one of these possible Token sequences:\n 1. [?foo]\n 2. [?bar]\n 3. [LParen]\n 4. [Star]\nbut found: '1'",
"token": {
"image": "1",
"startOffset": 47,
"endOffset": 47,
"startLine": 3,
"endLine": 3,
"startColumn": 10,
"endColumn": 10,
"tokenTypeIdx": 7,
"tokenType": {
"PATTERN": {},
"tokenTypeIdx": 7,
"CATEGORIES": [],
"categoryMatches": [],
"categoryMatchesMap": {},
"tokenName": "INTEGER",
"isParent": false
}
},
"resyncedTokens": [],
"context": {
"ruleStack": [
"SparqlDoc",
"QueryUnit",
"Query",
"SelectQuery",
"SelectClause"
],
"ruleOccurrenceStack": [
0,
0,
0,
0,
0
]
}
}
],
"cst": {
"name": "SparqlDoc",
"children": {
"Prologue": [
{
"name": "Prologue",
"children": {
"PrefixDecl": [
{
"name": "PrefixDecl",
/* . . . etc. . . . */
Check out our comprehensive API docs for more information.
After cloning the repo, in the project's root, run:
yarn install
All development-relevant code, including tests, is located in the repo's src/
directory. All of this code should be written in TypeScript. After making
changes, you will want to run our test scripts (run by Jest), like so:
yarn test
Stylistic conventions and formatting are enforced via tslint and prettier,
which means you don't really need to worry about those things yourself
(both the linter and prettier will check and format your code automatically
on any commit). If you'd like to force prettier to format your code (a la
gofmt), just run:
yarn format
To have the project run a lint check, tests, and compilation while you make changes, simply run (and leave running):
yarn dev
The preferred way of making a pull request is as follows:
bug/104-turtle-prefix-error).If you don't have contributor rights, you will need to fork the repo before doing steps 2 and 3, and then submit the PR from your fork.
Be sure to run yarn docs before submitting your PR, so that our type docs are
updated accordingly.
Copyright 2018 Stardog Union
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
FAQs
The Stardog Whisperer (error-tolerant parsers for languages used with Stardog)
The npm package millan receives a total of 113 weekly downloads. As such, millan popularity was classified as not popular.
We found that millan demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
Threat actors hijacked Toptal’s GitHub org, publishing npm packages with malicious payloads that steal tokens and attempt to wipe victim systems.
Research
/Security News
Socket researchers investigate 4 malicious npm and PyPI packages with 56,000+ downloads that install surveillance malware.
Security News
The ongoing npm phishing campaign escalates as attackers hijack the popular 'is' package, embedding malware in multiple versions.