floca

FLOCA

Enterprise-grade microservice architecture for NodeJS

========

floca is a complete solution to aid the development of enterprise-grade applications following the microservices architectural pattern. A proven stack used by numerous of systems in production for years now.

The basic idea of floca is to provide an abstraction level allowing your project to be freed of technical layers and unnecessary decoration. Your codebase will we required to define only pure CommonJS modules and all underlaying services will be set up by floca configured by a simple JSON object.

Features

• Proven, exceptionally featureful microservices architecture
• REST and Websockets as endpoints
• Internal services to be called by in-house entities only
• Tracking: you can monitor every message delivered (request or response) by only few lines of code
• Flow control / Reproducibility: A flow of communication / messages can be halted / continued / reinitiated anytime with no effort
• Advanced routing & listening: system fragmentation, qualified names, regular expressions, wildcards, etc.
• Channel-agnostic you can use any underlaying technology your application requires: AMQP, XMPP, etc...
• JSON Web Tokens integrated
• Built-in logging service supporting Loggly as well
• Short learning curve: no need to learn hundred of pages, communication has to be simple after all. CommonJS is all you need to understand. :)

System model

Of course, you can orchestrate your app the way you prefer, but please let us share a possible way to build up the codebase of an EE app using floca:

Each microservice

• has an own repository
• possesses only just a few CommonJS definition and optional configuration file(s)
• developed by a dedicated developer.
• is functional on its own. Can be built and deployed to a cloud provider or docker image with ease.

Your code is clean, pure and surprisingly simple.

Installation

$ npm install -g floca

Quick setup

Using the CLI tool:

$ floca create demoApp
$ cd demoApp
$ npm install
$ npm start

The CLI tool will create a new folder 'demoApp', and will create all required subfolders and files in it. By executing those npm commands, your microservice is ready-to-serve! The initial project will contain:

• a microservice
• a Starter code
• a preliminary package.json
• a sample config for the microservices

Creating own project:

var Floca = require('floca');

var floca = new Floca({
	entities: {
		appName: 'NameOfYourApp',
		serviceName: 'NameOfYourMicroService'
	}
});

floca.start( function(){
	console.log('Started.');
} );

Yes, it is that simple.

Note: by default floca is using in-memory bus to relay messages. Should you consider an enterprise-grade provider, should the AMQP or NSQ topic reviewed.

---

In either case, the app will

• read and watch your 'bus' folder and all entities defined in it
• publish entities and endpoints you defined
• be ready to serve :)

Usage

• Microservice configuration See for details
• AMQP See for details
• NSQ See for details
• Logging See for details
• JSON Web Tokens See for details
• Extender functions See for details
• Own SSL certification See for details
• Service configuration via service discovery See for details
• CLI See for details

Microservice configuration

Each microservice must have a unique name and must belong to an application or domain. These names will be used in the message bus for routing and distinguish different fields of jurisdiction.

{
	...
	entities: {
		folder: path.join( process.cwd(), 'bus'),
		appName: 'APP_NAME_IS_MISSING',
		serviceName: 'SERVICE_NAME_IS_MISSING',
		configurator: 'Tuner'
	}
	...
}

Microservices will be read from the folder defined by the attribute 'folder'. An inner entity, called 'Publisher' watches that folder and according to the changes at file-level, the micoservice entities will be republished or removed. The folder 'bus' in the root of your project is the default spot.

The attributes 'appName' and 'serviceName' are used to identify the app and the microservice and to identify the constructs in the message bus.

if the attribute 'configurator' is present, the Publisher will use to get the initial configuration of the microservices before publishing. This way, you can have a centralised configuration microservice with a name identified by the attribute 'configurator' and all microservices within the same app can reach it and be configured by the object such microservices retrieves. The Publisher will send a message to the configurator microservice passing the appName and serviceName and accept the JS object interpreted as configuration and passed to the microservice.

AMQP

An enterprise-grade message bus is highly required for a microservice architecture. AMQP has been proven probably the most reliable message solution out there. A stunning wide range of features support your aims. If you generated your project using the CLI with the option --amqp, then your project is set. If not, please make sure you create and pass the provider to floca:

var Fuser = require('floca');
var FuserAMQP = require('floca-amqp');
...

var fuserAMQP = new FuserAMQP();
var fuser = new Fuser( _.assign( {
	channeller: fuserAMQP,
	...
}, require('./config') ) );

This will tell floca to use the AMQP provider instead of the default in-memory solution.

floca tries to access a running AMQP by default specified in the config file as below:

{
	...
	amqp: {
		connectURL: 'amqp://localhost'
	}
	...
}

You can set an empty string to make floca work 'offline' or specify a valid connectionURL fitting your environment.

If the following environment variables are set, floca will respect them as configuration:

AMQP_CONN_URL

NSQ

If you generated your project using the CLI with the option --nsq, then your project is set. If not, please make sure you create and pass the provider to floca:

var Fuser = require('floca');
var FuserNSQ = require('floca-nsq');
...

var fuserNSQ = new FuserNSQ();
var fuser = new Fuser( _.assign( {
	channeller: fuserNSQ,
	...
}, require('./config') ) );

This will tell floca to use the NSQ provider instead of the default in-memory solution.

floca checks for settings for NSQ messaging solution in the config file as below:

{
	...
	nsq: {
		nsqdHost: '127.0.0.1'
		nsqdPort: 4150
	}
	...
}

If the following environment variables are set, floca will respect them as configuration:

NSQ_HOST, NSQ_PORT

Logging

floca supports mutliple logging solution:

• own logging service
• loggly
• file-based logging using winston

Please see the options in priority order:

Own logging service

var logger = ...;
{
	log: logger
	...
}

By setting the 'log' attribute to an own logger object possessing a 'log' function, floca will use it for all internal logging activity.

Loggly

{
	log: {
		loggly: {
			token: '',
			subdomain: ''
		}
	}
	...
}

If 'loggly' attribute is present with filled values, floca will establish connection to the Loggly server and use it as logging facility.

If the following environment variables are set, floca will respect them as configuration:

LOGGLY_TOKEN, LOGGLY_SUBDOMAIN

Log to files

{
	log: {
		level: "info",
		file: "./floca.log",
		maxsize: 10485760,
		maxFiles: 10
	}
	...
}

As third option, you can log into files (maybe to a shared drive on a VM accumulated by some background service) and configure the size, number of the files specifying the minimum level of interest in logging records.

!Note: support for other logging solutions is in the pipeline...

Back to Usage

JSON Web Tokens

JWT is an open industry standard method for representing claims securely between two parties. floca has built-in support for JWT. You can activate it by adding secret key to the configuration:

	...
	server: {
		...
		jwt: {
			key: 'x-floca-jwt',
			secret: '',
			timeout: 2 * 24 * 60 * 60,
			acquire: true
		},
		...
	}
	...

The attribute 'key' will be the key to be read from the header and will be added to the response as 'Access-Control-*' headers. To support token requests, the optional attribute 'acquireURI' will activate a REST request on the given URI allowing clients to acquire a token using a simple GET request.

Back to Usage

Extenders

floca allows you to insert extender functions to the setup process to extend, refine or overwrite the default behavior of floca. Extender functions have to be put to the config file. You might not need all of them, probably none of them. See an example config file below:

{
	connectMiddlewares: function( ){ ... },
	extendPureREST: function( config, app, pathToIgnore, harcon, tools ){ ... },
	extendREST: function( config, rester, pathToIgnore, harcon, tools ){ ... },
	runDevelopmentTest: function( rester, harcon ){ ... }
}

connectMiddlewares

Connect delivers the HTTP server framework for floca, and when it is initiated, the function connectMiddlewares is searched for to extend the middleware list of connect. By default only middlewares compression and body-parser is used. Your neeed might evolve the presence of other middlewares like helmet if web pages must be provided. Your function connectMiddlewares should return an array of middlewares to be used.

	connectMiddlewares: function( ){
		return [ helmet.xframe('deny') ];
	}

extendPureREST

The very pure way to define REST middlewares for auth libraries like Passport and any kind of low-level solution.

	extendPureREST: function( config, app, pathToIgnore, harcon, tools ){
		app.get('/auth/provider', passport.authenticate('provider') );
	}

extendREST

The exceptionally featureful connect-rest is used as REST layer in floca. The default behavior of floca publishes the microservices - if needed - and might publish the JWT request function. Your project might require to extend it with new REST functions like login:

	extendREST: function( config, rester, pathToIgnore, harcon, tools ){
		rester.post( { path: '/login', context: '/sys', version: '1.0.0' }, function( request, content, callback ){
			harcon.ignite( null, '', 'DBServices.login', content.email, content.password, function(err, user){
				if( err ) return callback( err );

				sendJWTBack( user[0].uid, user[0].roles, options, callback );
			} );
		}, { options: true } );
	}

Tools is an object enlisting services like JWT used by the floca.

runDevelopmentTest

floca can be started in development mode using the '--development' switch in the command line. This activates a scheduled checkup function printing out the list of published microservices and the method 'runDevelopmentTest' if present.

	runDevelopmentTest: function( rester, harcon ){
		harcon.simpleIgnite( 'DBServices.addAdmin', function(err, res){ } );
	}

You might want to create documents, user records at startup time for development purposes...

Back to Usage

SSL

If you possess an own SSL key, you can specify the absolute path of the key,cert file pairs in the configuration file as below:

	...
	server: {
		...
		ssl: {
			key: path.join( process.cwd(), 'ssh', 'sign.key' ),
			cert: path.join( process.cwd(), 'ssh', 'sign.crt' )
		},
		...
	}
	...

Back to Usage

Service configuration via service discovery

In a highly fragmented system, the configuration management should be centralised and accessed through service discovery.

	...
	entities: {
		configurator: 'Tuner'
	},
	...

The attribute 'configurator' will activate the configuration management in floca and the microservice loader will call the function 'config' of it passing the name of the app and the service to require the configuration sent to the entity to initialise.

CLI

floca is delivered with an embedded CLI tool to aid project creation and management.

Install floca with -g switch:

$ npm install -g floca

This will give you the floca command line statement

Generate Project

To generate a new project execute the following:

$ floca create <projectName> [--amqp] [--nsq]

This will create a folder projectName inside the execution folder and creates a minimal viable floca project using the transport provider you might pass. The project can be used right away:

$ npm install
$ npm start

Extend the bus folder with entities and have a happy coding!

Generate Test Mocha code

If your project possesses the service entities you might want to use, the CLI tool can generate Mocha tests for them:

$ floca generate test --mocha

This will execute a floca instance and all entities providing REST or Websocket interface will be associated with a test case.

All code will be put to the file: test/mochaTest

Execute this statement to call mocha on tests:

$ mocha test/mochaTest

Tests are generated with always accept behavior waiting for being unfold.

License

(The MIT License)

Copyright (c) 2015 Upwards Motion Ltd (1st Floor, 2 Woodberry Grove, Finchley, London N12 0DR; Company Registration No: 09074890)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Bugs

See https://github.com/UpwardsMotion/floca/issues.

Changelog

• 0.5.0: JWT integration added.
• 0.3.0: CLI tool added.
• 0.1.0: First version released.