rdfa-streaming-parser
Advanced tools
A fast and lightweight streaming and 100% spec-compliant RDFa 1.1 parser, with RDFJS representations of RDF terms, quads and triples.
The streaming nature allows triples to be emitted as soon as possible, and documents larger than memory to be parsed.
$ npm install rdfa-streaming-parser
or
$ yarn add rdfa-streaming-parser
This package also works out-of-the-box in browsers via tools such as webpack and browserify.
import {RdfaParser} from "rdfa-streaming-parser";
or
const RdfaParser = require("rdfa-streaming-parser").RdfaParser;
RdfaParser is a Node Transform stream
that takes in chunks of RDFa data,
and outputs RDFJS-compliant quads.
It can be used to pipe streams to,
or you can write strings into the parser directly.
While not required, it is advised to specify the profile of the parser
by supplying a contentType or profile constructor option.
const myParser = new RdfaParser({ baseIRI: 'https://www.rubensworks.net/', contentType: 'text/html' });
fs.createReadStream('index.html')
.pipe(myParser)
.on('data', console.log)
.on('error', console.error)
.on('end', () => console.log('All triples were parsed!'));
const myParser = new RdfaParser({ baseIRI: 'https://www.rubensworks.net/', contentType: 'text/html' });
myParser
.on('data', console.log)
.on('error', console.error)
.on('end', () => console.log('All triples were parsed!'));
myParser.write('<?xml version="1.0"?>');
myParser.write(`<!DOCTYPE html>
<html>
<head prefix="foaf: http://xmlns.com/foaf/0.1/">`);
myParser.write(`<link rel="foaf:primaryTopic foaf:maker" href="https://www.rubensworks.net/#me" />`);
myParser.write(`</head>`);
myParser.write(`<body>`);
myParser.write(`</body>`);
myParser.write(`</html>`);
myParser.end();
This parser implements the RDFJS Sink interface,
which makes it possible to alternatively parse streams using the import method.
const myParser = new RdfaParser({ baseIRI: 'https://www.rubensworks.net/', contentType: 'text/html' });
const myTextStream = fs.createReadStream('index.html');
myParser.import(myTextStream)
.on('data', console.log)
.on('error', console.error)
.on('end', () => console.log('All triples were parsed!'));
Optionally, the following parameters can be set in the RdfaParser constructor:
dataFactory: A custom RDFJS DataFactory to construct terms and triples. (Default: require('@rdfjs/data-model'))baseIRI: An initial default base IRI. (Default: '')language: A default language for string literals. (Default: '')vocab: The initial vocabulary. (Default: '')defaultGraph: The default graph for constructing quads. (Default: defaultGraph())features: A hash of features that should be enabled. Defaults to the features defined by the profile. (Default: all features enabled)profile: The RDFa profile to use. (Default: profile with all features enabled)contentType: The content type of the document that should be parsed. This can be used as an alternative to the 'profile' option. (Default: profile with all features enabled)htmlParseListener: An optional listener for the internal HTML parse events, should implement IHtmlParseListener (Default: null)new RdfaParser({
dataFactory: require('@rdfjs/data-model'),
baseIRI: 'http://example.org/',
language: 'en-us',
vocab: 'http://example.org/myvocab',
defaultGraph: namedNode('http://example.org/graph'),
features: { langAttribute: true },
profile: 'html',
htmlParseListener: new MyHtmlListener(),
});
On top of RDFa Core 1.1, there are a few RDFa variants that add specific sets of rules, which are all supported in this library:
'html' profile with 'text/html' as content type.'xhtml' profile with 'application/xhtml+xml' as content type.'xml' profile with 'application/xml', 'text/xml' and 'image/svg+xml' as content types.This library offers three different ways to define the RDFa profile or setting features:
'text/html' to the contentType option in the constructor.'', 'core', 'html', 'xhtml' or 'svg' to the profile option in the constructor.features option in the constructor.The table below lists all possible RDFa features and in what profile they are available:
| Feature | Core | HTML | XHTML | XML | Description |
|---|---|---|---|---|---|
baseTag | ✓ | ✓ | If the baseIRI can be set via the <base> tag. | ||
xmlBase | ✓ | If the baseIRI can be set via the xml:base attribute. | |||
langAttribute | ✓ | ✓ | ✓ | If the language can be set via the language attribute. | |
onlyAllowUriRelRevIfProperty | ✓ | ✓ | ✓ | If non-CURIE and non-URI rel and rev have to be ignored if property is present. | |
inheritSubjectInHeadBody | ✓ | ✓ | If the new subject can be inherited from the parent object if we're inside <head> or <body> if the resource defines no new subject. | ||
datetimeAttribute | ✓ | ✓ | ✓ | If the datetime attribute must be interpreted as datetimes. | |
timeTag | ✓ | ✓ | ✓ | If the <time> tag contents should be interpreted as datetimes. | |
htmlDatatype | ✓ | ✓ | If rdf:HTML as datatype should cause tag contents to be serialized to text. | ||
copyRdfaPatterns | ✓ | ✓ | ✓ | If rdfa:copy property links can refer to rdfa:Pattern's for copying. | |
xmlnsPrefixMappings | ✓ | ✓ | ✓ | ✓ | If prefixes should be extracted from xmlns. |
skipHandlingXmlLiteralChildren | If children of rdf:XMLLiteral should not be handled as RDFa anymore. This is not part of the RDFa spec. | ||||
xhtmlInitialContext | ✓ | If the XHTML initial context should be included in the initial prefixes. | |||
roleAttribute | ✓ | ✓ | ✓ | If the role attribute should be handled. |
This tool makes use of the highly performant htmlparser2 library for parsing HTML in a streaming way. It listens to tag-events, and maintains the required tag metadata in a stack-based datastructure, which can then be emitted as triples as soon as possible.
Our algorithm closely resembles the suggested processing sequence, with a few minor changes to make it work in a streaming way.
If you want to make use of a different HTML/XML parser,
you can create a regular instance of RdfaParser,
and just call the following methods yourself directly:
onTagOpen(name: string, attributes: {[s: string]: string})onText(data: string)onTagClose()This parser passes all tests from the RDFa 1.1 test suite. More specifically, the following manifests are explicitly tested:
The following optional features for RDFa processors are supported:
The following optional features for RDFa processors are not supported (yet):
This software is written by Ruben Taelman.
This code is released under the MIT license.
FAQs
A fast and lightweight streaming RDFa parser
We found that rdfa-streaming-parser demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
RubyGems.org has added a new "maintainer" role that allows for publishing new versions of gems. This new permission type is aimed at improving security for gem owners and the service overall.
Security News
Node.js will be enforcing stricter semver-major PR policies a month before major releases to enhance stability and ensure reliable release candidates.
Security News
Research
Socket's threat research team has detected five malicious npm packages targeting Roblox developers, deploying malware to steal credentials and personal data.