carson

Carson

Carson is an execution engine capable of supervising benchmarks on any RDF Query engine. It can be used to measure performance, and various other metrics, on both clients and servers, and can be adapted to any RDF based system.

Why use Carson ?

When you want to perform a benchmark on a query engine, you often spend hours to build a workflow that can run your tests and record all the data you need across multiples servers. Carson allows you to skip this step and focus on what you want to benchmark instead of how to perform the benchmark.

Features

• Easy to configure: setup Carson for your system with one configuration file!
• Deploy reverse proxies on servers of a federation and perform measurements on any incoming request.
• Support many common metrics (execution time, answer's completeness, number of transferred tuples, etc).
• Written using ECMAScript 6.

Table of contents

• Installation
• Usage
• Getting started
• What Carson does not do
• Use Carson as a library
• Documentation
• Testing
• License
• Author

Prerequisite

Node.js: version 6.9.0 or higher

Installation

For a global installation (to use it in command line mode)

npm install -g carson

For a local installation (to use it as a library)

npm install --save carson

Usage

Carson can be used through command line (see below) or as a library (see this section).

Usage: carson <config-file> <output> [options]

  Supervise benchmarks of any RDF Query engine with ease

  Options:

    -h, --help      output usage information
    -V, --version   output the version number
    -p, --progress  display progress during execution

Getting started

To work, Carson only need one configuration file that describes your setup and the metrics you which to use. Let's take a simple example where we want to measure the execution time of one request made to an LDF server:

• First, we start a LDF server on port 5000 (see LDF documentation for this).
• We put a simple SPARQL query SELECT * WHERE { ?s ?p ?o .} in the file all.sparql.
• Next, we use the following configuration:

{
	"type": "http",
	"servers": [
		{
			"host": "localhost",
			"port": 5000,
			"proxyPort": 5001
		}
	],
	"input": {
		"type": "single",
		"value": "all.sparql"
	},
	"metrics": ["time"],
	"format": {
		"client": "application/sparql-results+json",
		"federation": "rdf"
	},
	"run": "ldf-client http://localhost:5001/dbpedia -f $INPUT -t application/sparql-results+json"
}

• Then, we launch Carson with our configuration file and the path where we want to place the results:

carson config.json out/results.json

• The results will look like this (with some variations around the execution time of course):

{
	"nbServers": 1,
	"nbQueries": 1,
	"query#1": {
		"time": [
			{
				"time": 115,
				"startTime": "11:28:7",
				"endTime": "11:28:7",
				"origin": "unknown",
				"target": {
					"port": 5000,
					"host": "localhost"
				}
			},
			{
				"time": 615,
				"startTime": "11:28:7",
				"endTime": "11:28:7",
				"origin": "client",
				"target": "federation"
			}
		]
	}
}

For more details about how to write a configuration file, please see the related documentation.

What Carson does not do

It's important to clarify what Carson is about: it's a tool which can supervise the execution of a benchmark setup for you. It will setup reverse proxies around the servers of your federation and run queries using specific commands, but you still need to take care of the following tasks:

• Provides the query engine you want to use
• Deploy the servers of your federation.
• Defines the queries used for the benchmark.
• Ensure that both clients & servers observe the format defined in the configuration file.

Use Carson as a library

You can also use Carson as a library and call it through your own code.

const Carson = require('carson');

const app = new Carson('path/to/output/results.json');
const config = {
  // some awesome configuration here!
};
app.loadConfig(config)
	.then(() => app.run())
	.then(() => console.log('Execution completed!'), error => console.error(error));

Documentation

Generate JSDoc documentation with the following command

cd carson/

# documentation will be generated in doc/ directory
npm run doc

Testing

cd carson/

# run tests
npm test

# calculate code coverage with istanbul
npm run coverage

License

MIT License

Author

Thomas Minier