rpc-websocket

#RPC WebSocket

1. Synopsis
2. Preamble
2.1. The short story
2.2. Definitions
3. Installation
4. Browser support
5. Examples
5.1. Running the examples
5.2. Example 1
5.3. Example 2
5.4. Example 3
5.5. Example 4
6. Events supported
6.1. Server
6.2. Socket
7. API
7.1. Socket
7.1.1. RpcSocket(webSocket)
7.1.2. send(messageType, userData)
7.1.3. rpc(messageType, userData, replyHandler)
7.1.4. close()
7.2. Server
7.2.1. RpcServer(server)
7.2.2. close()
8. Development tools
9. Building
10. Testing
11. Why RPC Socket
11.1. What is wrong with ajax
11.2. What is wrong with JSON-RPC
11.3. What is wrong with socket.io
12. Contact
12.1. Support
12.2. Projects
13. Other publications
14. License

##1. Synopsis RPC WebSocket is a wrapper for standard websockets that adds support for message types, RPC, and for before/after send/receive events. It is an alternative to ajax, socket.io, and JSON-RPC.

##2. Preamble

###2.1. The short story

Distributed computing allows for distributed lambdas (=functions). If invoking a local function, looks like this:

var y=f(x1,x2);

/* do something with y */

In that case, the same function invocation, as function doing the computation on a remote server, that is, a Remote Procedure Call (RPC) looks like that:

ws.rpc('f',[x1,x2],function(y) {

  /* do something with y */

 });

Rpc WebSocket allows you to make remote function calls over a websocket.

###2.2. Definitions

• message types: Each message is always assigned a type. This allows us to transparently route messages to different handler functions.
• RPC: This feature implements the ability to call functions on the server using websockets. It is an alternative to ajax and to JSON-RPC.
• before/after send/receive events: For the purposes of logging, encryption, and compression, we need the ability to intercept incoming and outgoing messages before they are delivered to their handler functions. Depending on the application, there may be other reasons to apply wholesale changes to each incoming or outgoing message. We could, for example, add validation logic before sending messages.

I have tested Rpc WebSocket with the engine.io and engine.io-client transport mechanisms, but you should most likely be able to use alternative websocket implementations.

##3. Installation

You can install RPC-WebSocket with npm:

npm install rpc-websocket

If you want to use engine.io on the server side as the transport layer, you can install it with npm:

npm install engine.io

On the client side you can use engine.io-client:

npm install engine.io-client

##4. Browser support The module comes with a browserified version:

    browser-support/rpc-websocket-bundle.js

And a minimized version thereof:

    browser-support/rpc-websocket-bundle.min.js

You can use RPC WebSocket directly in the browser like this:

var webSocket = new WebSocket('ws://html5rocks.websocket.org/echo');
var ws=new RpcSocket(webSocket);

In order to support older browsers, you may want to use something like the engine.io-client wrapper for the browser. In older browsers, it will simulate the websocket logic using older transmission mechanisms.

##5. Examples

5.1. Running the examples

Open two terminals. In order to run example 1, in one terminal, start the server:

cd myproject/node_modules/rpc-websocket
node doc/examples/1-send-server.js

In the other terminal, you can execute the client:

cd myproject/node_modules/rpc-websocket
node doc/examples/1-send-client.js

5.2. Example 1

For: Sending/receiving typed messages

Programs

The client

var ioSocket = require('engine.io-client')('ws://localhost:8081');
var RpcSocket=require('rpc-websocket');
var ws=new RpcSocket(ioSocket);

ws.on('open', function() {
    ws.send('test/mtype1','something');
    ws.send('test/mtype2','something');
});

ws.on('test/mtype1', function(data) {
        console.log(data);
        ws.close();
});

ws.on('test/mtype2', function(data) {
        console.log(data);
        ws.close();
});

The server

var engine = require('engine.io');
var server = engine.listen(8081);
var RpcServer=require('rpc-websocket').server;
var wss=new RpcServer(server);

console.log('server started ...');

wss.on('connection', function(ws) {

    ws.on('test/mtype1', function(message) {
        console.log('received: %s', message);
            ws.send('test/mtype1','something back');
    });

    ws.on('test/mtype2', function(message) {
        console.log('received: %s', message);
            ws.send('test/mtype2','something else back');
    });

});

Output

The client

    something back

The server

    server started ...
    received: something
    received: something

As you can see, you can just resort to a naming convention to create something like a test channel or namespace.

5.3. Example 2

For: Making RPC calls

You can let the client make RPC calls to the server, but you can also let the server make RPC calls to the client. Server-to-client RPC calls are not possible with ajax. The fact that this is not possible, endlessly complicates the construction of particular types of applications such as real-time chat boxes.

Programs

The client

var ioSocket = require('engine.io-client')('ws://localhost:8081');
var RpcSocket=require('rpc-websocket');
var ws=new RpcSocket(ioSocket);

ws.on('open', function() {

    ws.rpc('test-type','something',function(message){
        console.log('received the following reply:'+message);
        ws.close();
    });

});

The server

var engine = require('engine.io');
var server = engine.listen(8081);
var RpcServer=require('rpc-websocket').server;
var wss=new RpcServer(server);

console.log('server started ...');

wss.on('connection', function(ws) {

    ws.on('test-type', function(message,reply) {
        console.log('received: %s', message);
        reply('something back');
    });

});

Output

The client

    received the following reply:something back

The server

    server started ...
    received: something

5.4. Example 3

For: Handling before/after send/receive events

You can use the beforeSend event to make changes to the message that is about to be sent. You can use the afterSend event to do some logging, for example, after successfully sending a message. You can also use the beforeReceive and afterReceive events. Here an example:

Programs

The client

var ioSocket = require('engine.io-client')('ws://localhost:8081');
var RpcSocket=require('rpc-websocket');
var ws=new RpcSocket(ioSocket);

ws.on('open', function() {
    ws.send('test-type','something');
});

ws.on('test-type', function(data) {
        console.log(data);
});

ws.on('beforeSend',function(data) {
        data['data']='changed before sending';
        console.log('before sending:'+JSON.stringify(data));
});

ws.on('afterSend',function(data) {
        console.log('after sending:'+JSON.stringify(data));
});

ws.on('beforeReceive',function(data) {
        console.log('before receiving:'+JSON.stringify(data));
});

ws.on('afterReceive',function(data) {
        console.log('after receiving:'+JSON.stringify(data));
        ws.close();
});

The server

var engine = require('engine.io');
var server = engine.listen(8081);
var RpcServer=require('rpc-websocket').server;
var wss=new RpcServer(server);

console.log('server started ...');

wss.on('connection', function(ws) {
        ws.on('test-type', function(message) {
                console.log('received: %s', message);
        });

        ws.send('test-type','something back');

});

Output

The client

    before sending:{"data":"changed before sending","messageType":"test-type"}
    after sending:{"data":"changed before sending","messageType":"test-type"}
    before receiving:{"data":"something back","messageType":"test-type"}
    something back
    after receiving:{"data":"something back","messageType":"test-type"}

The server

    server started ...
    received: changed before sending

5.5. Example 4

For: Looping over RPC calls

You could easily run into very subtle bugs when you start looping over RPC calls. For example:

var assert=require('assert');
var ChildProcess=require('child_process');

describe('rpc', function(){
        it('should be able to send/receive rpc calls', function(){
        
                //start test server
                var child=ChildProcess.exec('./test/2-rpc-server.js');

                //arrange for a client
                var ioSocket = require('engine.io-client')('ws://localhost:8082');
                var RpcSocket=require('rpc-websocket');
                var ws=new RpcSocket(ioSocket);

                //count messages
                var arrivedCount=0;
                var MESSAGES_PER_TYPE=5;
                var MAX_MESSAGES=3*MESSAGES_PER_TYPE;

                function attemptToTerminate() {
                        arrivedCount++;
                        if(arrivedCount===MAX_MESSAGES) 
                                child.kill('SIGKILL');
                }

                //client sending/receiving
                ws.on('open', function() {
                        for(var i=0; i<MESSAGES_PER_TYPE; i++) {

                                (function(i) {
                                        ws.rpc('test-type1',i,function(message){
                                                assert.equal(message,'TEST-BACK-1-'+i);
                                                attemptToTerminate();                                        
                                        });

                                        ws.rpc('test-type2',i,function(message){
                                                assert.equal(message,'TEST-BACK-2-'+i);
                                                attemptToTerminate();                                        
                                        });

                                        ws.rpc('test-type3',i,function(message){
                                                assert.equal(message,'TEST-BACK-3-'+i);
                                                attemptToTerminate();                                        
                                        });
                                })(i);

                        }

                });

        });

});

While the program is executing the reply logic, the value of the loop's counter i is not what you may think it is. Since the reply logic gets executed asynchronously, your socket could be waiting for a while before getting a response. In the meanwhile your loop counter will have moved on.

If you loop over an RPC call, you can generally not count of the fact that the variables that went into the request, are still the same as when you started the RPC call. Therefore, you must make sure to permanently fix their values in an enclosure:

while(condition) {

        ws.rpc('f',[x1,x2],function(y) {

                (function(x1,x2) {

                        /* do something with y */

                })(x1,x2);

        });
}

When the enclosure function exits, the logic inside of the function will hang on to copies of x1, and x2 inside its function closure, with values as they were at the moment that the program finished executed the function. The technique to create such enclosure function is called IIFE (Immediately-invoked function expression).

##6. Events supported

###6.1. Server

• connection: no arguments

###6.2. Socket

By bubbling the events from the standard websocket transport layer:

• open: no arguments
• close: no arguments
• error: injects a standard javascript error object with message, type, and description fields

Rpc Socket events:

• beforeReceive: injects a network-level message argument
• afterReceive: injects a network-level message argument
• beforeSend: injects a network-level message argument
• afterSend: injects a network-level message argument

A network-level message is an object with the following fields:

• data : the user data
• messageType: the message type
• msgid: in case of an RPC, the message ID
• rpc: in case of an RPC, the role of this message in the rpc process: request or reply

##7. API

7.1. Socket

7.1.1. RpcSocket(webSocket)

Constructor. Wraps a websocket object.

Params

• object webSocket The websocket object to wrap.

7.1.2. send(messageType, userData)

Sends a message through a websocket.

Params

• string messageType The message's type.
• any userData The data to send.

7.1.3. rpc(messageType, userData, replyHandler)

Makes an RPC call through a websocket.

Params

• string messageType The message's type (=function name).
• any userData The data to send.
• function replyHandler The reply handler to call when the response arrives from the server.

7.1.4. close()

Closes the websocket.

7.2. Server

7.2.1. RpcServer(server)

Constructor. Wraps a websocket server object.

Params

• object server The server object to wrap.

7.2.2. close()

Closes the server object.

##8. Development tools

• Standard websockets: engine.io, engine.io-client
• browser support: browserify, uglifyjs
• Unit tests: mocha
• Documentation: markdox, markdown-pp.py

9. Building

Execute the build.sh script to re-build the project from sources. It will re-generate the browser support files and the documentation.

10. Testing

Open a terminal and use mocha to run the unit tests:

cd node_modules/rpc-websocket
mocha

The unit tests work by starting the client who will in turn spawn a server process. The unit tests are mostly complete now.

##11. Why RPC Socket

11.1. What is wrong with ajax

With nowadays half of the internet hanging together with ajax, it is easy to forget that ajax is just a hack in which we reuse the http protocol to do something that it was not designed for. Ajax is not particularly suitable as an RPC mechanism. But then again, since ajax was the only RPC-like mechanism that browsers until recently supported, ajax is indeed what we have used to build half of the existing internet.

11.2. What is wrong with JSON-RPC

JSON-RPC has made the same mistake as SOAP and XML-RPC. JSON-RPC inspects the messages being sent and forces the developer to conform to a particular arrangement or even to a formal schema. JSON-RPC adds a bureaucratic procedure at a point in time when most developers would rather remain in prototyping mode. It causes the following reaction: Get out of my way, because I am too busy for this right now. I've got other things on my mind.

There would be nothing wrong with adding structural validation logic before sending a message, but that is rather something for later on in the project. Furthermore, there are many ways to do that. One size will not fit all. In practice, as you can see from most REST APIs floating around on the web, most applications will simply not implement any formal validation schema system at all.

With RPC WebSocket, you can still send whatever you like, just like with standard websockets. It is just a router module that facilitates the delivery of messages to the right handlers inside your program.

11.3. What is wrong with socket.io

Somewhere in the future, there will probably be nothing wrong with socket.io. In this month of August 2014, there were at some point 600+ outstanding, unresolved issues. I personally also logged a trouble ticket for something that we can only call a bug, but I have not heard back from their helpdesk.

Socket.io supports lots of features on top of websockets, such as support for express and koa. They also implements numerous scenarios in which you can use websockets with namespaces. You can even join and leave rooms. I only needed the custom events (=message types) and acknowledgements (=rpc). In RPC Socket I did not want and did not implement namespaces, because you can just prefix your message types with a namespace in order to create separate channels in one websocket.

In RPC Socket I did not implement support for express or koa. You could as well run the websocket server on another port, or even in another virtual machine, if you are worried about firewalls. Mixing http traffic with websockets in one server process, looks like an excellent way to create an undebuggable monster.

If you combine the socket.io features in unexpected ways, you may be in for a surprise. Just for the hell of it, I tried to use namespaces combined with acknowledgements. The entire edifice came crashing down and my trouble ticket is still unanswered. By the way, I did not find any unit tests in the socket.io sources that test for such combination of features. There are many other scenarios possible for using websockets than the ones implemented today in socket.io. I wonder, however, are they going to keep adding every possible new scenario? That could only end in a fully-fledged disasterzilla ...

12. Contact

###12.1. Support For trouble tickets with RPC WebSocket, please, use the github issue list.

###12.2. Projects I am available for commercial projects.

In commercial projects, I often do the initial prototyping by myself. After that, I manage external developer contributions through github and bitbucket. I usually end up being the long-term go-to person for how to evolve the system. My work involves reviewing Javascript for both the web and nodejs. I occasionally still do PHP. The startups I work for, are usually located elsewhere, but I do all of my work from Cambodia. If you are in need of a source code manager or quality assurance person for your project, feel free to contact me at [e-mail].

##13. Other publications

• At github.com
• At www.npmjs.org

##14. License

    RPC Websocket
    Written by Erik Poupaert, Cambodia
    (c) 2014
    Licensed under the LGPL