harcon

Harcon - Proven and reliable microservice solution for the harmonic convergence of JS entities. Scalable from in-browser web components till highly structured node-based enterprise components of real-time systems.

======== harcon is a service bus for NodeJS/Browser giving superior abstraction layer for interoperability between entities in a highly structured and fragmented ecosystem. It allows you to design and implement complex workflows where context and causality of messages are important.

The library has a stunning feature list beyond basic messaging functionality.

• Channel-agnostic: harcon represents a very abstract messaging framework allowing you to use any underlaying technology your application requires: AMQP, ZeroMQ, XMPP, etc... For zeromq integration, please check this: harcon-zero

• Tracking: you can monitor every message delivered (request or response) by only few lines of code

• Flow control / Reproducibility: A flow of communication / messages can be halted / continued / reinitiated anytime with no effort

• Free orchestration: your system can be orchestrated and distributed as you wish, message delivery is not limited to nodes or hosts

• Short learning curve: no need to learn hundred of pages, communication has to be simple after all

• Transparent: although harcon introduces lots of complex types and structures, your code and callbacks will be kept clean and pure, everything is (un)packed in the background in a transparent way

• Smooth infiltration: your objects / functions will possess the necessary services via injection, no need to create complex structures and compounds

• Advanced routing & listening: system fragmentation, qualified names, regular expressions, wildcards, etc.

!Note: Harcon's concept is to introduce a clean and high abstraction layer over messaging between entities. Like in case of every abstraction tool, for webapps which are simple as 1, it can be proven as a liability.

!Note: To use in browser, a CommonJS-enabled packager has to be applied like browserify or webpack or jspm.

This library starts to shine in a highly structured and distributed environment.

Installation

$ npm install harcon

Quick setup

var Inflicter = require('Inflicter');
var inflicter = new Inflicter( );

// define a listener function listening every message related to "greet" like "greet.goodmorning" or "greet.goodday"
inflicter.addict( null, 'peter', 'greet.*', function(greetings1, greetings2, callback){
	callback(null, 'Hi there!');
} );

// define an plain object serving as listener withing the context "greet" to messages "warm"
marie = {
	name: 'marie',
	context: 'greet',
	warm: function(greetings1, greetings2, callback){
		callback( null, 'Bonjour!' );
	}
};
inflicter.addicts( marie );

// sends a communication 'greet.everyone' with parameters and defines a callback to handle responses
// will receive back 2 answers: 'Hi there!' and 'Bonjour!'
inflicter.simpleIgnite( 'greet.everyone', 'Whatsup?', 'How do you do?', function(err, res){
	console.log( err, res );
} );

Back to Feature list

Workflows

In an enterprise-level system, one has to realize complex communication structure where lots of entities are following business logic and rules, involving subsystems and external resources, policies and other considerations, in short form: workflows. I take the liberty to define the workflow now as well defined routes and causality of messages. Simple method calls do the same, you can say. Yes and no. In a workflow, you are not dependent on the response timeframe, workflows manage distance in time and space. The recepient of a message can be on another server or city or planet. Recepient can answer right away or tomorrow or never.

Let me show a very short example: You are a company providing VPN services to customers. Orders taken by agents go to some accounting and client management subsystem and eventually your subsystem dealing with the technical setup receives a request through an interface of yours. Next step is to identify the network the user will be connected to, so a message is sent to the networking department who will respond maybe a day later. When it does, you have to continue your workflow where it is stopped, so you try to allocate network resources there and if it is successful you create a network configuration firmware to be used on the client's router to communicate with your backbone. When it is done by a config creator submodule of yours, you send it to an operation department for testing and when it is done you send back the results to the accounting for validation. And of course everything must be traceable and reconstructable and maybe rollable backwards. This is an extremely simplified use case, in real-life, workflows are much more complicated things and even harder to handle properly.

harcon is not a workflow designer tool, "just" a low-level library to manage such processes. You define entities and the communications among them then publish them.

You can resurrect a workflow if it failed and continue where it failed. You have to understand some details to use this lib at full scale.

Entities

In harcon, the communication unit is called simple entity. One can define 2 type of entities:

• simple function: when you associate a function with an event-pattern. Recommended to be used as observer, job-like, surveillance-, or interface-related asset.

// Qualified name - will answer to only this message
inflicter.addict( null, 'hugh', 'allocate.ip', function(callback){
	callback(null, 'Done.');
} );
// Wildcards - will answer anything within the context greet
inflicter.addict( null, 'peter', 'greet.*', function(callback){
	callback(null, 'Done.');
} );
// Regular expression - will answer anything where message name start with string 'job'
inflicter.addict( null, 'john', /job.*/, function( partner, callback){
	callback(null, 'Done.');
} );
...
inflicter.simpleIgnite( 'job.order', {name: 'Stephen', customerID:123}, function(err, res){
	console.log( 'Finished.', err, res );
} );
inflicter.simpleIgnite( 'john.job', {name: 'Stephen', customerID:123}, function(err, res){
	console.log( 'Finished.', err, res );
} );

• objects: service object enclosing service functions as a unique and complete context. Recommended to be used as business entities.

var bookKeeper = {
	name: 'BookKeeper',
	...
	newOrder: function( customer, callback ){
		callback( null, 'Done.' );
	},
	ordersOfToday: function( callback ){
		callback( null, [] );
	}
};
...
inflicter.simpleIgnite( 'BookKeeper.newOrder', {name: 'Stephen', customerID:123}, function(err, res){
	console.log( 'Finished', err, res );
} );
inflicter.simpleIgnite( 'BookKeeper.ordersOfToday', function(err, res){
	console.log( 'Finished.', err, res );
} );

Both component types must possess a unique name serving as the simplest way to send a message to. Or just simple using the regexp addressing any service function matching the pattern...

Responses

By default, harcon returns and array of response objects returned by the entities addressed by a sent message.

Let's have 2 simple entities:

inflicter.addict( null, 'peter', 'greet.*', function(callback){
	callback(null, 'Hi.');
} );
inflicter.addict( null, 'camille', 'greet.*', function(callback){
	callback(null, 'Hello.');
} );

inflicter.simpleIgnite( 'greet.simple', function(err, res){
	console.error( err, res );
} );

Returns with the following:

[ 'Hi.', 'Hello.' ]

(Note: for consistency reason, you got an array even if one entity.)

In some cases, you might find useful to know which answer comes from which entity. If you add a single parameter to the harcon:

var inflicter = new Inflicter( { namedResponses: true } );

The returned object will look like this:

{ peter: 'Hi.', camille: 'Hello.' }

Your callback might receive an error object in unwanted situations. The default transport channel of harcon will stop the message processing at the first error occurring as follows:

inflicter.addict( null, 'peter', 'greet.*', function(callback){
	callback( new Error('Stay away, please.') );
} );
inflicter.addict( null, 'camille', 'greet.*', function(callback){
	callback( new Error('Do not bother me.') );
} );

inflicter.simpleIgnite( 'greet.simple', function(err, res){
	console.error( err, res );
} );

will result the following on your console:

[Error: Stay away, please.] null

The default transport layer is designed for development purposes or in-browser usage. In an EE, environment, please mind the introduction of a message queue solution like: AMQP, ZeroMQ.

For ZeroMQ, you can find an "official" plugin here: harcon-zero

By using different transport layer, all occurred error messages will be delegated. In such cases, harcon will retrieve an Error object encapsulating all error object received from entities.

Orchestration

Basically, you define service functions which can be called through its name (object-based entity) or expression evaluation (function-based entity). When you orchestrate a complex system, you define object-based entities providing functions to be called. There are 2 orthogonal ways to orchestrate such entities in your system.

Context: a qualified name identifying the field/purpose the entity is operating. For example an entity parsing incoming JSON document can have the context "transfer" answering communications addressed to "transfer.parse" where parse is the function provided by that entity. Within a given context, multiple entitiy can answer a communication with a given name.

var parser = {
	name: 'JSONParser',
	context: 'transfer',
	parse: function( document, callback ){
		callback( null, 'Done.' );
	}
};
var observer = {
	name: 'Observer',
	context: 'transfer',
	parse: function( document, callback ){
		callback( null, null );
	}
};

Sending a message "transfer.parse" will be interpreted as follows: context: "transfer" functionSelector: "parse" The entities published in the context "transfer" possessin the function "parse" will be notified and their service function will be invoked.

A context might contain subcontexts depending on the complexity of your system.

Division: divisions is a diffferent angle of orchestrating entities. A division is a closed "box" of entities, meaning that an entity can operate only within the division it is member of. Every entity belongs to a division. Divisions can be encapsulated, so a complete division-tree can be built-up in a harcon application. The reason why divisions are important, because it represents a responsibility unit. Entities within it (in normal cases) cannot see outside and an entity published to a container division can answer to messages initiated by an entity somewhere lower in the tree. This gives you a control to define surveillance-like or control-like features and much higher complexity of communication-management.

Note: these features are not mandatory to be used. The complexity will tell you how to orchestrate. If you only need function-based simple entities, feel free to go along with them. If you need to implement a highly structured money transaction management system in a financial environment, those features above will be urged to be defined.

Chain messages

To chain messages, define the next point in the workflow you have to add another parameter to your service function:

var order = {
	name: 'Order',
	context: 'order',
	newVPN: function( customer, ignite, callback ){
		ignite( 'allocate.address', '127.0.0.1', function(err, res){
			callback(err, res);
		} );
	}
};
...
inflicter.simpleIgnite( 'order.newVPN', {name: 'Stephen', customerID:123}, function(err, res){
	console.log( 'Finished', err, res );
} );

That will initiate a small workflow. inflicter.simpleIgnite sends a message to entity Order who will send within the same workflow to the Allocator. When it answeres, then the message of the beginning will be answered. harcon will know if you initiate a message within the processing of another one and considers it as part of the ongoing workflow and tracks it. Mind the async execution to keep everything in track!

Call back or not?

You are not forced to always send answer, in some cases a quite entities is desired. If you do not define a callback neither side of the communication, harcon will consider it as a one-ways message sending.

// Qualified name - will answer to only this message
inflicter.addict( null, 'karl', 'reserve.address', function( address ){
	// Do something...
} );
...
inflicter.simpleIgnite( 'reserve.address', '127.0.0.1' );

Entity initialization

The need to pass contextual parameters to entities might rise. The options object passed to the constructure of Inflicter allows you to specify parameters for entities which will be passed while the init method defined in the entity is called.

inflicter = new Inflicter( { /* ... */ marie: {greetings: 'Hi!'} } );
var marie = {
	name: 'marie',
	context: 'test',
	init: function (options) {
		// {greetings: 'Hi!'} will be passed
	}
	// services ...
};

Logging

When you create the Inflicter instance, you can pass a logger object which will be respected and used to do loggings. If not set, harcon will log everything to the console. So in production, setting up a logging facility ( like winston or bunyan ) is strongly adviced.

inflicter = new Inflicter( { logger: logger /* ... */ } );

That logger instance will be used as logging facility all over the system, including internal services and entities. Each entity receives a function: harconlog with the signature:

function( err, message, obj, level ){ ... }

That function can be used anything within your entity object:

var Marie = {
	name: 'Marie',
	context: 'greet',
	whiny: function (greetings, callback) {
		this.harconlog( null, 'Some logging', { data: greetings }, 'silly' );
		callback( null, 'Pas du tout!' );
	}
};

That function should be used for any logging activity you need during the flow of your app.

Unique messages

Every communication exchanged possesses the following properties (not exclusively):

• unique ID
• reference to the parent message if exists
• uniqued ID of the workflow itself
• external ID of the workflow started by an external communication involving a reference number to consider
• timestamp

Any time you sends a message or receives an answer, such objects are bypassing through the harcon system which logs and tracks all of them.

By default, harcon uses 32 as length of the IDs which are unique over time and among computer nodes. You can override this default when initiating Inflicter

inflicter = new Inflicter( { /* ... */ idLength: 32 } );

Message exchange

When you defined your components, the need to send and receive messages arises. In a workflow, your component might initiate a message, response one or while responding one sends other ones. The function-based components can perform only the latter 2 cases, cannot initiate anything by its own. This type of components are present to define services, listeners, definitely not serious business entities. As you saw above, the serices functions might possess a parameter before the callback: ignite

var order = {
	name: 'Order',
	context: 'order',
	newVPN: function( customer, ignite, callback ){
		ignite( 'allocate.address', '127.0.0.1', function(err, res){
			callback(err, res);
		} );
	}
};

That ignite can be used to chain messages, which means to send messages during the processing of a received one. The tool to initiate sub-workflows.

Of course components are not just reacting entities, they might launch new workflows as well. Object-based components possesses an injected function: ignite and can be used as follows:

var timer = {
	name: 'Timer',
	scheduling: function( ){
		this.ignite( 'validate.accounts', function(err, res){
		} );
	}
};

That ignite function is injected by the harcon when you publish the components.

Divisions

Systems can be orchastrated into divisions which is a tree structure actually. One can create divisions following the control-flow or responsibility-chain of the application. Every component you deploy will belong to a division. If not declared, then to the system division where all system-level components are put. Divisions is not just a logical grouping of components, but also an encapsulation-model. A component cannot send messages outside the own division but can send to the inner ones. This means, that system components can send to any component, but non-system components cannot reach the level of the main system or other branches of the division-tree.

Divisions give you a very easy-to-use structure to orchestrate your system. Of course, you can use the harcon without using divisions, the complexity of your system will show if you needed it or not.

Let's define components and add them to divisions:

// This will add John to the division 'workers'
inflicter.addict( 'workers', 'john', /job.*/, function(callback){
	callback(null, 'Done.');
} );
// This will add Claire to the division 'entrance'
var claire = {
	name: 'claire',
	division: 'entrance',
	context: 'greet',
	simple: function (greetings1, greetings2, callback) {
		callback(null, 'Enchanté, mon plaisir!');
	}
};

Components in a division can be called to:

inflicter.ignite( null, 'entrance', 'greet.simple', 'Hi', 'Ca vas?', function(err, res){
} );

Note: please keep in mind, that inflicter.ignite can be and should be used only when you initiate a workflow from outside the harcon!

If you inititate a communication through the inflicter instance, it means, that you wants to drop in a message "from outside" which could mean an integration with an external system or just kick off a workflow. There are 2 methods to be called:

ignite and simpleIgnite

The different between them is the parameter list. The later does not require to specify external messageId or division, the prior one does as you can see in the example just above.

External ID is very useful, when the workflow is initiated by some external event possessing an id which must be kept for further logging or tracking or just because a communication harmonization across the complete company.

Entity configuration

Entities can be configured easily. The init method of an entity if present, will be called at the end of its publishing process withing the harcon. The function can be used to set up DB connections or anything required for your entities.

module.exports = {
	name: 'Claire',
	context: 'greet',
	init: function (options, callback) {
		console.log('Init...', options);
		callback();
	}
};

The method receives a configuration object and a callback to be called when the init method finishes. That configuration object can be passed when an entity is published:

inflicter.addicts( Claire, config );

or even before, when harcon is created.

inflicter = new Inflicter( { logger: logger, idLength: 32, Claire: {greetings: 'Hi!'} } );

Extension

harcon can be easily extended by using pure harcon components listening to system events:

var extension = {
	name: 'As you design it',
	context: inflicter.name,
	castOf: function( name, firestarter ){
	},
	affiliate: function( firestarter ){
	},
	close: function(){
	}
}
inflicter.addicts( extension );

In the current version, the inflicter instance you are using will send to your components events about system closing, entity publishing and revoking. For a working example, please check harcon-radiation.

License

(The MIT License)

Copyright (c) 2015 Imre Fazekas

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Bugs

See https://github.com/imrefazekas/harcon/issues.

Changelog

• 1.0.X : serious fixes
• 1.0.0 : first stable release
• 0.9.0 : small redesign to allow to use in a Browserify/Webpack environment
• 0.8.0 : automated (re/un)deployment added, rewritten event coordination subsystem
• 0.6.0 : delivery fixes
• 0.5.0 : initial release