Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
barnard59
Advanced tools
Readme
barnard59
is a toolkit to automate extract, transform and load (ETL) tasks. Its main focus is on creating Linked Data. It allows you to generate RDF out of non-RDF data sources. In doing so, it follows the standard adopted in Semantic Web.
More specifically, barnard59
is an engine to execute data pipelines.
A pipeline is a way to extract, transform, and load data (ETL). It allows you to consolidate data from various sources, and publish it as RDF. Using a pipeline you can:
barnard59
expects a pipeline to be defined as linked-data. The supported formats are:
application/ld+json
application/n-triples
text/turtle
application/rdf+xml
To learn more about building your own pipeline, go here.
To execute pipeline with barnard59
, you will need node.js
environment. All node
and npm
versions which are actively maintained are supported by barnard59
.
To check if you have Node.js installed, run this command in your terminal:
node -v
To confirm that you have npm installed, run:
npm -v
If you need to install (or upgrade) node, or npm, go here.
Once node and npm are installed, you can install the dependencies for this project. To do so, run:
npm install
To execute pipeline, run:
barnard59 run <pipeline_file>
The following arguments are available:
--pipeline <pipeline_iri>
--output <filename>
--verbose
or -v
--enable-buffer-monitor
--variable <name=value>
If the file contains more than one pipeline, you can define the pipeline to execute. By default, first pipeline will be executed.
To define pipeline to be executed, run:
barnard59 run <pipeline_file> --pipeline <pipeline_iri>
Some pipelines will produce a write stream. By default, this stream can be will be printed in the terminal. To write it to the file, run:
barnard59 run <pipeline_file> --output <output_file>
Pipelines can represent complex data manipulation. To better understand what is happening, add options:
--verbose
or -v
- to enable diagnostic console output--enable-buffer-monitor
- to enable histogram of buffer usageIt is possible to pass arguments from command line to the pipeline. This allows you to use a variable in the pipeline, whithout defining its value. The key-value pairs for pipeline variables can be provided at the runtime. In the cli tool, this can be done via the --variable
argument.
To pass an argument to the pipeline, run:
barnard59 run <pipeline_file> --variable <variable_name_in_pipeline>=<value>
For example, to pass example.txt
as a filename
to pipeline.json
, run:
barnard59 run pipeline.json --variable filename=example.txt
In a CI/CD environment you might want to define sensitive arguments like usernames and passwords as environment variables. To pass them to the pipeline, run:
barnard59 run pipeline.json --variable password=$PASSWORD
where $PASSWORD
is a an environment variable.
To pass multiple arguments to the pipeline, call --variable
argument multiple times:
barnard59 run <pipeline_file> --variable <variable1>=<value1> --variable <variable2>=<value2> ... --variable <variableN>=<valueN>
For example, to pass $USER
as a user
, and $PASSWORD
as a password
to pipeline.json
, run:
barnard59 run pipeline.json --variable user=$USER --variable password=$PASSWORD
This pipeline parses a CSV file. It uses the CSV on the Web format. The file examples/parse-csvw.ttl
contains two pipelines:
<parseCsvw>
- the main pipeline<parseMetadata>
- the metadata pipelineTho execute the main pipeline, run:
node bin/barnard59.js run examples/parse-csvw.ttl --pipeline=http://example.org/pipeline/parseCsvw
Or, if you'd like to use npx you can run:
npx barnard59 run examples/parse-csvw.ttl --pipeline=http://example.org/pipeline/parseCsvw
This pipeline downloads, and transforms json file. It:
context
variable in the JSON file.For comparison, the same pipeline is defined in JSON-LD and Turtle format.
To execute the JSON-LD pipeline, run:
node bin/barnard59.js run examples/fetch-json-to-ntriples.json --pipeline http://example.org/pipeline/cet
with npx:
npx barnard59 run examples/fetch-json-to-ntriples.json --pipeline http://example.org/pipeline/cet
The --pipeline
parameter is required, as the file contains two pipelines.
To execute the Turtle pipeline, run:
node bin/barnard59.js run examples/fetch-json-to-ntriples.ttl --pipeline http://example.org/pipeline/utc
or
npx barnard59 run examples/fetch-json-to-ntriples.ttl --pipeline http://example.org/pipeline/utc
By default, the pipeline stream will be written to stdout
. Use --output
parameter to write output to the file:
node bin/barnard59.js run examples/fetch-json-to-ntriples.json --pipeline http://example.org/pipeline/cet --output test.nt
or
npx barnard59 run examples/fetch-json-to-ntriples.json --pipeline http://example.org/pipeline/cet --output test.nt
FAQs
Barnard59 Linked Data pipelines
The npm package barnard59 receives a total of 2,309 weekly downloads. As such, barnard59 popularity was classified as popular.
We found that barnard59 demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.