floca

FLOCA

Enterprise-grade microservice architecture for NodeJS

========

floca is a complete solution to aid the development of enterprise-grade applications following the microservices architectural pattern. A proven stack used by numerous of systems in production for years now.

The basic idea of floca is to provide an abstraction level allowing your project to be freed of technical layers and unnecessary decoration. Your codebase will we required to define only pure CommonJS modules and all underlaying services will be set up by floca configured by a simple JSON object.

Features

• Proven, exceptionally featureful microservices architecture
• REST and Websockets as endpoints
• Internal services to be called by in-house entities only
• Tracking: you can monitor every message delivered (request or response) by only few lines of code
• Flow control / Reproducibility: A flow of communication / messages can be halted / continued / reinitiated anytime with no effort
• Advanced routing & listening: system fragmentation, qualified names, regular expressions, wildcards, etc.
• Channel-agnostic you can use any underlaying technology your application requires: AMQP, XMPP, etc...
• JSON Web Tokens integrated
• Built-in logging service supporting Loggly and Papertrail as well
• Short learning curve: no need to learn hundred of pages, communication has to be simple after all. CommonJS is all you need to understand. :)

System model

Of course, you can orchestrate your app the way you prefer, but please let us share a possible way to build up the codebase of an EE app using floca:

Each microservice

• has an own repository
• possesses only just a few CommonJS definition and optional configuration file(s)
• developed by a dedicated developer.
• is functional on its own. Can be built and deployed to a cloud provider or docker image with ease.

Your code is clean, pure and surprisingly simple.

Installation

$ npm install floca

Quick setup

Creating own project:

var Floca = require('floca');

var floca = new Floca({
	floca: {
		appName: 'NameOfYourApp',
		entityName: 'NameOfYourMicroService'
	}
});

floca.start( function(){
	console.log('Started.');
} );

Yes, it is that simple.

Note: by default floca is using in-memory bus to relay messages. Should you consider an enterprise-grade provider, should the AMQP or NSQ topic reviewed.

---

In either case, the app will

• read and watch your 'bus' folder and all entities defined in it
• publish entities and endpoints you defined
• be ready to serve :)

Usage

• Microservice configuration See for details
• AMQP See for details
• NSQ See for details
• Logging See for details
• JSON Web Tokens See for details
• Extender functions See for details
• Own SSL certification See for details
• Service configuration via service discovery See for details
• CLI See for details

Microservice configuration

Each microservice must have a unique name and must belong to an application or domain. These names will be used in the message bus for routing and distinguish different fields of jurisdiction.

{
	...
	floca: {
		folder: path.join( process.cwd(), 'bus'),
		appName: 'APP_NAME_IS_MISSING',
		entityName: 'ENTITY_NAME_IS_MISSING',
		configurator: 'Tuner'
	}
	...
}

Microservices will be read from the folder defined by the attribute 'folder'. An inner entity, called 'Publisher' watches that folder and according to the changes at file-level, the micoservice entities will be republished or removed. The folder 'bus' in the root of your project is the default spot.

The attributes 'appName' and 'entityName' are used to identify the app and the microservice and to identify the constructs in the message bus.

if the attribute 'configurator' is present, the Publisher will use to get the initial configuration of the microservices before publishing. This way, you can have a centralised configuration microservice with a name identified by the attribute 'configurator' and all microservices within the same app can reach it and be configured by the object such microservices retrieves. The Publisher will send a message to the configurator microservice passing the appName and entityName and accept the JS object interpreted as configuration and passed to the microservice.

AMQP

An enterprise-grade message bus is highly required for a microservice architecture. AMQP has been proven probably the most reliable message solution out there. A stunning wide range of features support your aims. If you generated your project using the CLI with the option --amqp, then your project is set. If not, please make sure you create and pass the provider to floca:

var Fuser = require('floca');
var FuserAMQP = require('floca-amqp');
...

var fuserAMQP = new FuserAMQP();
var fuser = new Fuser( _.assign( {
	channeller: fuserAMQP,
	...
}, require('./config') ) );

This will tell floca to use the AMQP provider instead of the default in-memory solution.

floca tries to access a running AMQP by default specified in the config file as below:

{
	...
	amqp: {
		connectURL: 'amqp://localhost'
	}
	...
}

You can set an empty string to make floca work 'offline' or specify a valid connectionURL fitting your environment.

If the following environment variables are set, floca will respect them as configuration:

AMQP_CONN_URL

NSQ

If you generated your project using the CLI with the option --nsq, then your project is set. If not, please make sure you create and pass the provider to floca:

var Fuser = require('floca');
var FuserNSQ = require('floca-nsq');
...

var fuserNSQ = new FuserNSQ();
var fuser = new Fuser( _.assign( {
	channeller: fuserNSQ,
	...
}, require('./config') ) );

This will tell floca to use the NSQ provider instead of the default in-memory solution.

floca checks for settings for NSQ messaging solution in the config file as below:

{
	...
	nsq: {
		nsqdHost: '127.0.0.1'
		nsqdPort: 4150
	}
	...
}

If the following environment variables are set, floca will respect them as configuration:

NSQ_HOST, NSQ_PORT

Logging

floca supports mutliple logging solution:

• own logging service
• loggly
• Papertrail
• file-based logging using winston

Please see the options in priority order:

Own logging service

var logger = ...;
{
	log: logger
	...
}

By setting the 'log' attribute to an own logger object possessing a 'log' function, floca will use it for all internal logging activity.

Loggly

{
	log: {
		loggly: {
			token: '',
			subdomain: ''
		}
	}
	...
}

If 'loggly' attribute is present with filled values, floca will establish connection to the Loggly server and use it as logging facility.

If the following environment variables are set, floca will respect them as configuration:

LOGGLY_TOKEN, LOGGLY_SUBDOMAIN

Papertrail

	{
		log: {
			papertrail: {
				host: 'logX.papertrailapp.com',
				port: '12345'
			}
		}
		...
	}

If the configuration file possesses a 'papertrail' attribute as shown above, floca will send the logging info to your papertrail account.

The following environment variables are respected:

PAPERTRAIL_HOST, PAPERTRAIL_PORT

Log to files

{
	log: {
		level: "info",
		file: "./floca.log",
		maxsize: 10485760,
		maxFiles: 10
	}
	...
}

As third option, you can log into files (maybe to a shared drive on a VM accumulated by some background service) and configure the size, number of the files specifying the minimum level of interest in logging records.

!Note: support for other logging solutions is in the pipeline...

Back to Usage

JSON Web Tokens

JWT is an open industry standard method for representing claims securely between two parties. floca has built-in support for JWT. You can activate it by adding secret key to the configuration:

	...
	server: {
		...
		jwt: {
			key: 'x-floca-jwt',
			secret: '',
			timeout: 2 * 24 * 60 * 60,
			acquire: true
		},
		...
	}
	...

The attribute 'key' will be the key to be read from the header and will be added to the response as 'Access-Control-*' headers. To support token requests, the optional attribute 'acquireURI' will activate a REST request on the given URI allowing clients to acquire a token using a simple GET request.

Back to Usage

Extenders

floca allows you to insert extender functions to the setup process to extend, refine or overwrite the default behavior of floca. Extender functions have to be put to the config file. You might not need all of them, probably none of them. See an example config file below:

{
	connectMiddlewares: function( ){ ... },
	extendPureREST: function( config, app, pathToIgnore, harcon, tools ){ ... },
	extendREST: function( config, rester, pathToIgnore, harcon, tools ){ ... },
	runDevelopmentTest: function( rester, harcon ){ ... }
}

connectMiddlewares

Connect delivers the HTTP server framework for floca, and when it is initiated, the function connectMiddlewares is searched for to extend the middleware list of connect. By default only middlewares compression and body-parser is used. Your neeed might evolve the presence of other middlewares like helmet if web pages must be provided. Your function connectMiddlewares should return an array of middlewares to be used.

	connectMiddlewares: function( ){
		return [ helmet.xframe('deny') ];
	}

extendPureREST

The very pure way to define REST middlewares for auth libraries like Passport and any kind of low-level solution.

	extendPureREST: function( config, app, pathToIgnore, harcon, tools ){
		app.get('/auth/provider', passport.authenticate('provider') );
	}

extendREST

The exceptionally featureful connect-rest is used as REST layer in floca. The default behavior of floca publishes the microservices - if needed - and might publish the JWT request function. Your project might require to extend it with new REST functions like login:

	extendREST: function( config, rester, pathToIgnore, harcon, tools ){
		rester.post( { path: '/login', context: '/sys', version: '1.0.0' }, function( request, content, callback ){
			harcon.ignite( null, '', 'DBServices.login', content.email, content.password, function(err, user){
				if( err ) return callback( err );

				sendJWTBack( user[0].uid, user[0].roles, options, callback );
			} );
		}, { options: true } );
	}

Tools is an object enlisting services like JWT used by the floca.

runDevelopmentTest

floca can be started in development mode using the '--development' switch in the command line. This activates a scheduled checkup function printing out the list of published microservices and the method 'runDevelopmentTest' if present.

	runDevelopmentTest: function( rester, harcon ){
		harcon.simpleIgnite( 'DBServices.addAdmin', function(err, res){ } );
	}

You might want to create documents, user records at startup time for development purposes...

Back to Usage

SSL

If you possess an own SSL key, you can specify the absolute path of the key,cert file pairs in the configuration file as below:

	...
	server: {
		...
		ssl: {
			key: path.join( process.cwd(), 'ssh', 'sign.key' ),
			cert: path.join( process.cwd(), 'ssh', 'sign.crt' ),
			ca:  path.join( process.cwd(), 'ssh', 'ca.pem' ),
			requestCert: true || false,
			rejectUnauthorized: true || false
		},
		...
	}
	...

Back to Usage

Service configuration via service discovery

In a highly fragmented system, the configuration management should be centralised and accessed through service discovery.

	...
	floca: {
		configurator: 'Tuner'
	},
	...

The attribute 'configurator' will activate the configuration management in floca and the microservice loader will call the function 'config' of it passing the name of the app and the service to require the configuration sent to the entity to initialise.

Bugs

See https://github.com/UpwardsMotion/floca/issues.

Changelog

• 0.20.0: Big steps toward 1.0
• 0.9.0: Tons of fixes and improvements
• 0.5.0: JWT integration added.
• 0.3.0: CLI tool added.
• 0.1.0: First version released.