koreografeye

Koreografeye

This is a miniature implementation of an choreographer implementing the Orchestrator for a decentralized Web network specification using an N3 reaoner (such as EYE) as internal rule engine.

Koreografeye was created to facilitate running automated processes against Solid pods. Our main use case is monitoring the LDN Inbox of Solid Pods for new notifications and running scripts when new data arrives.

Architecture

Koreografeye can be run on an input directory containing one or more RDF files (in Turtle or N3 format). For each of these files one or more N3 rule scripts can be executed. The results of this execution will be put in an output directory. If the N3 output contains Policy fragments, they can be executed using a policy executor.

Koreografeye is mainly targeted for processing Activity Streams (AS2) notifications. For each of these notifications N3 rules express one or more Policies what execution steps should be executed when receiving a AS2 notification of a particular shape. The orchestrator component reads all incoming AS2 notifications and executes the EYE reasoner (or EYE-JS, Roxi,... via configuration) on each of them and writes the results to an output directory. Policies are implemented as JavaScript plugins and can be execute by a policy executor.

A Policy is an RDF snippet that explains what should happen when running N3 rules against an AS2 input resource is successfull. Each policy has a name (identifier) and zero or more instances of an fno:Execution, which defines what JavaScript implementation should be executed. Optionally zero or more execution parameters can be provided for each execution.

The example below is an example of a Policy (prefix declarations omitted):

  ex:MyDemoPolicy pol:policy [
      a fno:Execution ;
      fno:executes ex:helloWorld ;
      ex:text "Hello, world!" ;
  ] .

This policy requests the execution of a ex:helloWorld implementation with one parameter ex:text with value "Hello, world!".

The ex:MyDemoPolicy can be the result of an N3 rule (prefix declarations omitted):

{
  ?id a as:Announce.
}
=>
{
  ex:MyDemoPolicy pol:policy [
      a fno:Execution ;
      fno:executes ex:helloWorld ;
      ex:text "Hello, world!" ;
  ] .
}.

The N3 above states: "If we find an AS2 notification of type as:Announce, then request executing the ex:MyDemoPolicy.

Installation

npm install koreografeye -g

The experimental blogic , policy and solid examples require the Prolog version of EYE installed. We refer to the install instructions of this project for more information.

Usage

Command line

• Put ActivityStreams notifications in the in directory
• Put N3 rules in the rules directory
• Run npx orch --info --keep rules/* --in in --out out to run the rules on alle in notification in the in directory
  • Use the --info option to show informational messages
  • Use the --keep option if you don't want to automatic clean the in after processing notifications
  • The processed notifications will end up in the out directory
  • Alternative: run npm run orch
• Run npx pol --info --keep --in out to run the policy executor with all the processed notifications from the out directory
  • Use the --keep option if you don't want to automatic clean the out after processing notifications
  • Alternative: run npm run pol
• If you want to experiment with expressing N3 rules as RDF Surfaces use the following commands:
  • run npm run orch:blogic : this will execute the rules/blogic/00_demo.n3 rule which is a direct translation of rules/00_demo.n3 in the RDF Surfaces language
  • run npm run orch:policy : this will read rules/blogic/00_policy.n3 (written in a small DSL language) and compile these rules into RDF Surfaces using a compiler available in rules/blogic/policy
• If you want to experiment with running N3 for montitoring resources on the web use the following commands:
  • Copy data/solid.jsonld to the in directory
  • run npm run orch:solid : this will check if a new resource actually exists on a backend server and send a Ntfy notifications (when running npm pol)

Typescript/javascript

Small javascript example to execute Koreografeye using demo.ttl and 00_demo.n3.

const { EyeJsReasoner } = require('./dist/orchestrator/reasoner/EyeJsReasoner')
const { executePolicies } = require('./dist/policy/Executor');
const { parseAsN3Store, readText, storeAddPredicate, makeComponentsManager } = require('./dist/util');

const store = await parseAsN3Store('./data/demo.ttl'); // input graph
const rules = [readText('./rules/00_demo.n3')]; // array of n3 rules serialized as string

// add main subject and origin for the reasoner
const mainSubject = 'urn:uuid:42D2F3DC-0770-4F47-BF37-4F01E0382E32';
storeAddPredicate(store, 'https://www.example.org/ns/policy#mainSubject', mainSubject);
storeAddPredicate(store, 'https://www.example.org/ns/policy#origin', './data/demo.ttl');

// execute reasoning (orchestration)
const reasoner = new EyeJsReasoner([ "--quiet" , "--nope" , "--pass"])
const reasoningResult = await reasoner.reason(store, rules);

const manager = await makeComponentsManager('./config.jsonld','.');

// execute policies
await executePolicies(manager, reasoningResult);

Note: for this code to run, the project has to be compiled first (npm run build).

Commands

orch

Run the orchstrator with one or more N3 rule files on an input directory of notifications.

bin/orch [options] rule [rule ...]

Options:

• --config file : orchestrator configuration file
• --in directory : directory with input notifications
• --out directory : directory with orchestrator output
• --err directory : directory with failed notifications
• --keep : keep the --in data (don't delete after processing)

pol

Run a policy executor on one of the output files of the orchestrator

bin/pol [options] file

Options:

• --config file : orchestrator configuration file
• --info : verbose messages
• --debug : debug messages
• --trace : trace messages

Plugins

A Koreografeye implements the side-effects for the policy executor. These plugins handle sending emails, updating Solid Pod, sending notifications, start engines and fire rockets. The plugins require an entry in the config.jsonld Components.js configuration file and a JavaScript/Typescript implementation.

Each Koreografeye plugin is a javascript class that extends PolicyPlugin and should implement a policyTarget function with the following signature:

export async function policyTarget(mainStore: N3.Store, policyStore: N3.Store , policy: IPolicyType) : Promise<boolean>;

where:

• mainStore: an N3.Store containing the parsed input file (without the policies)
• policyStore: an N3.Store containing the current active parsed policy
• policy: an IPolicyType structure containg all parameters for the requested policy:
  • node : N3.NamedNode | N3.BlankNode : the policy node ()
  • path : string : path to the input file
  • policy : string : identifier of the policy instance
  • target : string : identifier of the policy target
  • mainSubject : string : the activity identifier of the notification
  • origin : string : path to the original notification input file
  • args : any { [key: string] : RDF.Term | undefined } : key/value pairs of all arguments that were provided for the policy

An example IPolicyType contents (in JSON):

After running rules/00_demo.n3 on data/demo.ttl

{
    "node": {
        "termType": "NamedNode",
        "value": "http://example.org/MyDemoPolicy"
    },
    "path": "out/demo.ttl",
    "policy": "bc_0_b1_b0_bn_1",
    "target": "http://example.org/demoPlugin",
    "args": {
        "http://www.w3.org/1999/02/22-rdf-syntax-ns#type": {
            "termType": "NamedNode",
            "value": "https://w3id.org/function/ontology#Execution"
        },
        "https://w3id.org/function/ontology#executes": {
            "termType": "NamedNode",
            "value": "http://example.org/demoPlugin"
        },
        "http://example.org/param1": {
            "termType": "Literal",
            "value": "[e-mail].i",
            "language": "",
            "datatype": {
                "termType": "NamedNode",
                "value": "http://www.w3.org/2001/XMLSchema#string"
            }
        },
        "http://example.org/param2": {
            "termType": "Literal",
            "value": "[e-mail]",
            "language": "",
            "datatype": {
                "termType": "NamedNode",
                "value": "http://www.w3.org/2001/XMLSchema#string"
            }
        },
        "http://example.org/body": {
            "termType": "NamedNode",
            "value": "urn:uuid:42D2F3DC-0770-4F47-BF37-4F01E0382E32"
        }
    },
    "mainSubject": "urn:uuid:42D2F3DC-0770-4F47-BF37-4F01E0382E32",
    "origin": "file:///var/folders/g8/czx2gjfs3_bbvvk1hjlcsj9m0000gn/T/in/demo.ttl"
}

Known issues

• Policies must be provided using named nodes:

correct

 ex:MyDemoPolicy pol:policy [
      a fno:Execution ;
      fno:executes ex:demoPlugin ;
      ex:param1 "[e-mail].i" ;
      ex:param2 "[e-mail]" ;
  ] .

incorrect

 [ pol:policy [
      a fno:Execution ;
      fno:executes ex:demoPlugin ;
      ex:param1 "[e-mail].i" ;
      ex:param2 "[e-mail]" ;
      ex:subject "A new resource was created!" ;
      ex:body ?s 
  ] ] .

Examples

• KoreografeyeDemo : A demonstration koreografeye project
• KoreografeyePluginDemo : A demonstration how to create new plugins for koreografeye
• Koreografeye-Solid : A plugin that can update a Solid pod
• Koreografeye-Mastodon : A plugin that can send a toot to a Mastodon user
• OAI-Bridge : A service that turns OAI-PMH services into Event Notifications services
• Solid-Agent : Koreografeye as part of a rule-based intelligent software agent for Solid pods.