diogenes

Diogenes

When asked why he went about with a lamp in broad daylight, Diogenes confessed, "I am looking for a [honest] man."

Diogenes defines and executes functions with a common interface (services) configured in a directed acyclic graph.

What is a service

I define a "service" as a function with a specific interface. Its arguments are:

• a configuration, common to all services
• a list of dependencies (the output of other services)
• a callback (services are asynchronous by default)

A service outputs a "dependency", this is identified with a name. Services are organized inside registries. The common interface allows to automate how the dependencies are resolved within the registry.

From functions to services

Let's say that you have a function returning an html page. You usually need to execute a certain number of steps (already incapsulated into functions):

decodeURL(url, function (err, id){
  if (err) {
    returnHTML('Error');
  }
  getDB(config, function (err, db){
    if (err) {
      returnHTML('Error');
    }
    getDataFromDB(id, function (err, obj){
      if (err) {
        returnHTML('Error');
      }
      retrieveTemplate("template.html", function (err, template){
        if (err) {
          returnHTML('Error');
        }
        renderTemplate(template, obj, function (err, html){
          if (err) {
            returnHTML('Error');
          }
          returnHTML(html)
        });
      });
    });
  });
});

I am sure you have already seen something like this. Well, I can see more than one issue here. The first one, usually called "the pyramid of doom", can be solved easily using promises (or other techniques). But there is a worst issue, you are designing how the components interact between them, in an imperative way. This is awkward as you'll either use the same patterns again and again, or you'll spend a lot of time refactoring the old code trying to avoid repetition.

With Diogenes you can describe the flow of informations in terms of services, describing the relations between them:

var Diogenes = require('diogenes');
var registry = Diogenes.getRegistry();

registry.add("id", decodeURL);
registry.add("db",  getDB);
registry.add("data", ["db", "url"], getDataFromDB);// arrays define dependencies
registry.add("template", retrieveTemplate);
registry.add("html", ["template", "data"], renderTemplate);

and let the system do the job:

registry.run("html", configuration, returnHTML);

Diogenes resolves the whole dependency tree for you, executing the services in the right order (even in parallel when possible). Then it serves you the result on a silver platter.

A step by step example

Importing diogenes

The easiest way to import Diogenes is using commonjs:

var Diogenes = require('diogenes');

You can also import it as a global module. In that case you should take care of the dependencies (setImmediate and occamsrazor).

Creating a registry

You can create a registry with:

var registry = Diogenes.getRegistry();

Without arguments you create a "local registry" that is reachable within the scope of the "registry" variable. If you pass a name to the constructor you create a global registry that is available everywhere:

var registry = Diogenes.getRegistry("myregistry");

Defining services

A service is defined by a name (a string), a list of dependencies (an optional list of strings) and a function with a specific interface:

registry.add("text", function (config, deps, next) {
  var text = ["Diogenes became notorious for his philosophical ",
      "stunts such as carrying a lamp in the daytime, claiming to ",
      "be looking for an honest man."].join();
  next(undefined, text);
});

If the service is successful it passes undefined as the first argument and the result as second. The first argument will contain an exception if the service fails As an alternative it is also possible to return a value instead of using a callback or returning a promise. It will work anyway:

registry.add("text", function (config, deps) {
  return ["Diogenes became notorious for his philosophical ",
      "stunts such as carrying a lamp in the daytime, claiming to ",
      "be looking for an honest man."].join();
});

In this case you can throw an exception in case of errors. Let's add another service:

registry.add("tokens", ['text'], function (config, deps, next) {
  next(undefined, deps.text.split(' '));
});

The array specifies a list of dependencies. This service depends on the "text" service. The deps argument will contain an attribute for every dependency in this example: deps.text.

registry.add("count", ['tokens'], function (config, deps, next) {
  next(undefined, deps.tokens.length);
});

registry.add("abstract", ['tokens'], function (config, deps, next) {
  var len = config.abstractLen;
  var ellipsis = config.abstractEllipsis;
  next(undefined, deps.tokens.slice(0, len).join(' ') + ellipsis);
});

The "config" argument is the same for all services. It is passed with the run method below.

registry.add("paragraph", ['text', 'abstract', 'count'],
  function (config, deps, next) {
    next(undefined, {
        count: deps.count,
        abstract: deps.abstract,
        text: deps.text
    });
  });

This is how services relates each other:

Calling a service

You can call a service using the method "run" with the name and the configuration (the same one will be passed as argument to all services).

registry.run("paragraph", {abstractLen: 5, abstractEllipsis: "..."},
  function (err, p){
    if (err){
        console.log("Something went wrong!");
    }
    else {
        console.log("This paragraph is " + p.count + " words long");
        console.log("The abstract is: " + p.anstract);
        console.log("This is the original text:");
        console.log(p.text);            
    }
  });

p will be the output of the paragraph service. If any service throws, or returns an error, the "err" argument will contain the exception. If you need more than one service, you can pass a list of services:

registry.run(["count", "abstract"], {abstractLen: 5, abstractEllipsis: "..."},
  function (err, deps){
    ...
  });

In this case the second argument will contain an object with an attribute for any dependency (deps.count, deps.abstract). Using "run", Diogenes calls all services required to satisfy the dependencies tree. You can get the ordering using:

registry.getExecutionOrder("paragraph",
  {abstractLen: 5, abstractEllipsis: "..."});

It will return an array: ["text", "tokens", "abstract", "count", "paragraph"] Diogenes does not strictly follow that order: "count", for example doesn't require to wait for "abstract" as it depends on "tokens" only.

Plugins

A service can contain more than one function. The correct function will be chosen using the configuration and an occamsrazor validator. Diogenes.validator is a copy of occamsrazor.validator (for convenience). Let's say for example that you want to use a different way to get the abstract:

var useAlternativeClamp = Diogenes.validator().match({abstractClamp: "chars"});

registry.add("abstract", ['text'], useAlternativeClamp,
  function (config, deps, next) {
    var len = config.abstractLen;
    var ellipsis = config.abstractEllipsis;
    next(undefined, deps.text.slice(0, len) + ellipsis);
  });

You should notice that specifying a validator you are also able to use a different set of dependencies.

registry.getExecutionOrder("paragraph",
  {abstractLen: 5, abstractEllipsis: "...", abstractClamp: "chars"});

will output: ["text", "abstract", "tokens", "count", "paragraph"]. You can run the service as usual:

registry.run("paragraph",
  {abstractLen: 5, abstractEllipsis: "...", abstractClamp: "chars"},
  function (err, p){
    if (err){
        console.log("Something went wrong!");
    }
    else {
        console.log("This paragraph is " + p.count + " words long");
        console.log("The abstract is: " + p.anstract);
        console.log("This is the original text:");
        console.log(p.text);
    }
  });

The key point is that you just extended the system without changing the original code!

Add a value

addValue is a short cut method you can use if a service returns always the same value (or a singleton object). And it doesn't need any configuration. For example the "text" service can become:

registry.addValue("text", ["Diogenes became notorious for his philosophical ",
  "stunts such as carrying a lamp in the daytime, claiming to ",
  "be looking for an honest man."].join());

Cache a service

If the result of a service depends on the configuration, or it is heavy to compute, you can cache it. You can enable the cache with cacheOn, empty the cache with cacheReset or disable with cacheOff. The cacheOn method takes an object as argument with 3 different arguments:

• key: (a string/an array or a function) it generates the key to use as cache key. You can specify an attribute of the configuration (string), an nested property (array) or use a custom function running on the configuration. It default to a single key (it will store a single value)
• maxAge: the time in ms for preserving the cache. Default to infinity.
• maxSize: the length of the cache. Default to infinity

Note: a cache hit, will ever never return dependencies. After all if the service has a defined return value it doesn't need to relay on any other service. So for example:

registry.service('count').cacheOn({key: "abstractLen", maxAge: 1000});

Errors and fallback

If a service returns or throws an exception, this is propagated along the execution graph. Services depending on those are not executed. They are considered failed. While this is the default behaviour, it is also possible to configure a service to fallback on a default value:

registry.service('count').onErrorReturn(42);

Or on a function (the usual config is the only argument)

registry.service('count').onErrorExecute(function (config){
  return config.defaultCount;
});

This function is called in these cases:

• the "count" service thrown an exception
• the "count" service returned an exception
• one of the dependencies of the "count" service propagated an exception

Events

The event system allows to do something when a service is executed. You can listen to a service in this way:

registry.on('paragraph', function (name, dep, config){
  // name is "paragraph"
  // dep is the output of the "count" service
  // config is the usual one used in the "run" method
});

The event system is implemented with occamsrazor (see the doc, especially the "mediator" example https://github.com/sithmel/occamsrazor.js#implementing-a-mediator-with-occamsrazor). So you can execute the function depending on the arguments (just pass as many validators you need).

registry.on(function (name, dep, config){
  // this is executed for any service
});

registry.on("paragraph", isLessThan5, useAlternativeClamp, function (name, dep, config){
  // this is executed for count service
  // only if count is less than 5 and
  // the config passes the "useAlternativeClamp" validator
});

registry.on(/count.*/, function (name, dep, config){
  // this is executed for any service with the name that matches
  // that regular expression
});

Be aware that events are suppressed for cached values and their dependencies! You can also handle the event once with "one" and remove the event handler with "off". If you need you can also emit your own custom events:

registry.on("my custom event", function (name, data1, data2){
  //
});

registry.trigger("my custom event", data1, data2);

metadata

You can store any data related to a service with the metadata:

var service = registry.service("abstract")

service.metadata({abstractLen: 10});

registry.add("abstract", ['tokens'], function (config, deps, next) {
  var len = this.service('abstract').metadata().abstractLen;
  var ellipsis = config.abstractEllipsis;
  next(undefined, deps.tokens.slice(0, len).join(' ') + ellipsis);
});

This can be practical if you want to save informations that are "service" specific.

Documentation

You can attach a a description to a service. This will be used by the method "info" for giving an outline of the services available.

var service = registry.service("abstract")

service.description("This service returns the abstract of a paragraph.");
service.info({}); // I pass the configuration

abstract
========
This service returns the abstract of a paragraph.
Dependencies:
* text
* tokens

You can also use the method "info" of the registry to get all the services.

Dependencies

Diogenes depends on setimmediate and occamsrazor.

How does it work

A lot of the things going on requires a bit of knowledge of occamsrazor (https://github.com/sithmel/occamsrazor.js). Basically a service is an occamsrazor adapter's registry (identified with a name). When you add a function you are adding an adapter to the registry. This adapter will return the function and the dependencies when called with the configuration as argument. When you try running a service the first thing that happen is that diogenes will perform a dfs within the services. The configuration will be used to unwrap a service when its adjancency is required. Doing this operation you have this cases:

• the result is not defined because there is no function (or no function matching the configuration) attached to it. If this dependency is necessary it will generate an exception.
• the result is ambiguous as more than one adapter matches the configuration with the same validator's score. If this dependency is necessary it will generate an exception.
• the result matches a cache entry. The resulting adapter will return the cached value. There will be no dependencies.
• the result matches a function/dependencies

The result will be a sorted list of adapters. At this point the system will start executing all the functions. Every time one of these function's callback returns a value I push this in an dependency map and try to execute all the functions that see their dependencies fulfilled. The last function should be the one I requested.

Diogenes is ES6 friendly!

Using Diogenes with ES6 helps a lot if you want a concise syntax. But pay attention! you can't use "this" with arrow functions!

// using arrow function
service.add("textfile",
  (config, deps, next) => fs.readFile(config.path, 'utf8', next));

// destructuring and promises
service.add("user", (config, deps, next) => {
  const {id, firstName, lastName} = config;
  return axios.put('/user/' + id, {
    firstName: 'Fred',
    lastName: 'Flintstone'
  });
});

Syntax

Diogenes.getRegistry

Create a registry of services:

var registry = Diogenes.getRegistry();

or

var registry = new Diogenes();

If you don't pass any argument this registry will be local. And you will need to reach the local variable "registry" to use its services. If you pass a string, the registry will use a global variable to store services:

var registry = Diogenes.getRegistry("myregistry");

or

var registry = new Diogenes("myregistry");

This is convenient if you want any application register its services to a specific registry.

Registry's methods

service

Returns a single service. It creates the service if it doesn't exist.

registry.service("name");

add

It adds a service to a registry. It has different signatures:

registry.add(name, func);   

registry.add(name, dependencies, func);   

registry.add(name, dependencies, validator, func);   

• The name (mandatory) is a string. It is the name of the service. A registry can have more than one service with the same name BUT they should validate alternatively (see the validator argument).
• dependencies: it is an array of strings. Every string is the name of a service. This should be executed and its output should be pushed in the function
• validator: it is an occamsrazor validator. (https://github.com/sithmel/occamsrazor.js). You can also pass a different value as explained in the "match" validator (https://github.com/sithmel/occamsrazor.js#occamsrazorvalidatormatch);
• The function (mandatory) is a function returning a service.

The function can have 2 different signatures: with callback (config, deps, next) or without (config, deps):

• "config" is a value passed to all services when "run" is invoked
• "deps" is an object. It has as many properties as the dependencies of this service. The attributes of deps have the same name of the respective dependency.
• "next" is the function called with the output of this service: next(undefined, output).
• If something goes wrong you can pass the error as first argument: next(new Error('Something wrong!')).

If you use the signature without "next" you can return the value using return, or throw an exception in case of errors. If you return a promise (A or A+) this will be automatically used:

registry.add("promise", function (config, deps) {
  return new Promise(function (resolve, reject){
    resolve("resolved!");
  });
});
registry.run("promise", function (err, dep){
  console.log(dep); // resolved!
})

It returns the registry.

addValue

It works the same as the add method but instead of adding a service It adds a value. This will be the dependency returned.

registry.addValue(name, value);   

registry.addValue(name, dependencies, value);   

registry.addValue(name, dependencies, validator, value);   

Note: having a value you don't need dependencies. They are still part of the signature of the method for consistency.

addOnce

It works the same as add but the service will be returned only one time.

registry.addOnce(name, func);   

registry.addOnce(name, dependencies, func);   

registry.addOnce(name, dependencies, validator, func);   

addValueOnce

It works the same as addValue but the service will be returned only one time.

registry.addValueOnce(name, value);   

registry.addValueOnce(name, dependencies, value);   

registry.addValueOnce(name, dependencies, validator, value);   

remove

It remove a service from the registry:

registry.remove(name);

It returns the registry.

run

It executes all the dependency tree required by the service and call the function. All the services are called using config:

registry.run(name, config, func);

The function takes 2 arguments:

• an error
• the value of the service required

You can also use the alternative syntax:

registry.run(names, config, func);

In this case "names" is an array of strings (the dependency you want to be returned). The callback will get as second argument an object with a property for any dependency returned.

The context (this) of this function is the registry itself.

It returns the registry.

cacheOff

It empties and disable the cache of all services.

cacheReset

It empties the cache of all services.

cachePause

It pauses the cache of all services.

cacheResume

Resume the cache of all services.

getExecutionOrder

Returns an array of services that should be executed with those arguments. The services are sorted by dependencies. It is not strictly the execution order as diogenes is able to execute services in parallel if possible. Also it will take into consideration what plugins match and caching (a cached items as no dependency!):

registry.getExecutionOrder(name, config);

init

Helper function. It runs a group of functions with the registry as "this". Useful for initializing the registry.

/*module1 fir example*/
module.exports = function (){
  this.add('service1', ...);
};
/*main*/
var module1 = require('module1');
var module2 = require('module2');
registry.init([module1, module2]);

forEach

It runs a callback for any service registered.

registry.forEach(function (service, name){
  // the service is also "this"
});

on

Attach an event handler. It triggers when an services gets a valid output. You can pass up to 3 validators and the function. The function takes 3 arguments:

• the name of the service
• the output of the service
• the config (used for running this service)

registry.on([validators], function (name, dep, config){
  ...
});

one

The same as "on". The function is executed only once.

off

Remove an event handler. It takes the previously registered function.

registry.off(func);

trigger

Trigger an event. You can use trigger with a bunch of arguments and, all handlers registered with "on" and "one" compatible with those will be called.

info

It returns a documentation of all services. It requires a configuration to resolve the dependencies.

registry.info(config);

Chaining

add (addValue, addValueOnce, addOnce), remove and run are chainable. So you can do for example:

registry.add("service1", service1)
.add("service1", service2);
.add("service3", ["service1", "service2"], myservice);

"this" binding

In the "add" and "run" callback the "this" is the registry itself. This simplify the case in which you want:

• manipulate the cache of a service (reset for example)
• communicate between services using the event system
• run another service

Example:

var c = 0;
registry.add('counter-button', function (config, deps, next){
  var registry = this;
  document.getElementById('inc').addEventListener("click", function (){
    c++;
    console.log(c);
  });
  registry.on("reset-event", function (){
    c = 0;
  });
  next();
});

registry.add('reset-button', function (config, deps, next){
  var registry = this;
  document.getElementById('reset').addEventListener("click", function (){
    registry.trigger("reset-event");
  });
  next();
});

registry.run(['counter-button', 'reset-button']);

Service's methods

You can get a service with the "service" method.

var service = registry.service("service1");

All the service methods returns a service instance so they can be chained.

add

The same as the add registry method:

service.add(func);   

service.add(dependencies, func);   

service.add(dependencies, validator, func);   

addValue

The same as the addValue registry method:

service.addValue(value);   

service.addValue(dependencies, value);   

service.addValue(dependencies, validator, value);   

addOnce

The same as the addOnce registry method:

service.addOnce(func);   

service.addOnce(dependencies, func);   

service.addOnce(dependencies, validator, func);   

addValueOnce

The same as the addValueOnce registry method:

service.addValueOnce(value);   

service.addValueOnce(dependencies, value);   

service.addValueOnce(dependencies, validator, value);   

remove

The same as the remove registry method:

service.remove();   

run

The same as the run registry method:

service.run(config, func);

onErrorReturn

If the service or one of the dependencies fails (thrown an exception) it returns "value" as fallback.

service.onErrorReturn(value);

onErrorExecute

If the service or one of the dependencies fails (thrown an exception) it uses the function to calculate a fallback.

service.onErrorExecute(function (config){
  return ...;
});

onErrorThrown

It reverts to the default behaviour: on error it propagates the error.

service.onErrorThrown();

cacheOn

Set the cache for this service on. It takes as argument the cache configuration:

service.cacheOn(config);

The configuration contains 3 parameters:

• key: (a string/an array or a function) it generates the key to use as cache key. You can specify an attribute of the configuration (string), an nested property (array) or use a custom function running on the configuration. It default to a single key (it will store a single value)
• maxAge: the time in ms for preserving the cache. Default to infinity.
• maxSize: the length of the cache. Default to infinity

cacheOff

It empties and disable the cache.

cacheReset

It empties the cache.

cachePause

It pauses the cache. The cache will work as usual but the cached result won't be used

cacheResume

Resume the cache.

on/one/off

Manage event handlers. It is a alternate syntax to the registry ones.

registry.service(name).on([validators], function (name, dep, config){
  ...
});

registry.service(name).one([validators], function (name, dep, config){
  ...
});

registry.service(name).off(func);

metadata

Get/set metadata on the service.

registry.service(name).metadata(metadata); // set
registry.service(name).metadata(); // get

description

Get/set a service description.

registry.service(name).description(metadata); // set
registry.service(name).description(); // get

info

It returns a documentation of the service. It requires a configuration to resolve the dependencies.

registry.service(name).info(config);

Errors in the services graph

The library is currently able to detect and throws exceptions in a few cases:

• circular dependencies
• missing dependencies (or incompatible plugin)
• more than one plug-in matches

These 3 exceptions are thrown by "getExecutionOrder". So it is very useful using this method to check if something is wrong in the graph configuration.

Tricks and tips

Where to apply side effects

Do not mutate the configuration argument! It is not meant to be changed during the execution. Instead you can apply side effects through a dependency. See the example of the expressjs middleware below.

Run a service defined in a closure

If you need to run a service that depends on some variable defined in a closure you can use this trick: define a local registry containing the "local" dependencies, merge together the main and the local registry (a new merged registry will be generated), run the service. This is an example using an expressjs middleware:

var express = require('express');
var app = express();
var Diogenes = require('diogenes');
var registry = new Diogenes();

registry.add('hello', ['req', 'res'], function (config, deps, next){
  var username = deps.req.query.username;
  deps.res.send('hello ' + username);
  next();
});

app.get('/', function(req, res){
  var localReg = new Diogenes();
  localReg.addValue('req', req);
  localReg.addValue('res', res);
  registry.merge(localReg).run('hello');
});

app.listen(3000);

Using events for disposing/cleaning up

A service can return a dependency that need to be disposed. In this case you can leverage the event system:

var registry = new Diogenes();
...
registry.add('database-connection', function (config, deps){
  var connection = ..... I get the connection here

  this.on('done', function (){
    connection.dispose();
  });
  next();
});

registry.run('main-service', function (err, dep){
  ...
  this.trigger('done');
});