NoFlo is a simple flow-based programming implementation for Node.js. From WikiPedia:
In computer science, flow-based programming (FBP) is a programming paradigm that defines applications as networks of "black box" processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally component-oriented.
Developers used to the Unix philosophy should be immediately familiar with FBP:
This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
It also fits well in Alan Kay's original idea of object-oriented programming:
I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages (so messaging came at the very beginning -- it took a while to see how to do messaging in a programming language efficiently enough to be useful).
NoFlo has been written in CoffeeScript for simplicity. The system is heavily inspired by J. Paul Morrison's book Flow-Based Programming.
Currently NoFlo is still in quite early stages. It has already been used in some real-world applications, but the small number of available components still limits the utility of the system.
NoFlo is available via NPM, so you can install it with:
$ npm install -g noflo
NoFlo requires a reasonably recent version of Node.js, and some npm packages. Ensure you have the coffee-script package installed (coffee command should be available on command line) and NoFlo checked out from Git. Build NoFlo with:
$ cake build
Then you can install everything needed by a simple:
$ npm link
NoFlo is available from GitHub under the MIT license.
Please refer to the CHANGES.md document.
There are two ways to use NoFlo:
noflo toolWhen you create a NoFlo graph, it doesn't do anything by itself. It only loads the components of the graph and sets up the connections between them. Then it is up to the components to actually start sending messages to their outports, or reacting to messages they receive on their inports.
Since most components require some input before they act, the usual way to make a NoFlo graph run is to send it some initial information packets, or IIPs. Examples of this would include sending a port number that a web server could listen to the web server component, or sending a file name to a file reader.
This activation model provides many possibilities:
File line count using embedded NoFlo:
$ coffee ./examples/linecount/count.coffee somefile.txt
File line count as an individual NoFlo application:
$ noflo -i
NoFlo>> load examples/linecount/count.json
or
$ noflo examples/linecount/count.json
Simple "Hello, world" web service with Basic authentication using embedded NoFlo:
$ coffee ./examples/http/hello.coffee
Then just point your browser to http://localhost:8003/. Note that this example needs to have connect NPM package installed. Username is user and password is pass.
A component is the main ingredient of flow-based programming. Component is a CommonJS module providing a set of input and output port handlers. These ports are used for connecting components to each other.
NoFlo processes (the boxes of a flow graph) are instances of a component, with the graph controlling connections between ports of components.
Since version 0.2.0, NoFlo has been able to utilize components shared via NPM packages. Read the introductory blog post to learn more.
Functionality a component provides:
Minimal component written in CoffeeScript would look like the following:
noflo = require "noflo"
class Forwarder extends noflo.Component
description: "This component receives data on a single input
port and sends the same data out to the output port"
constructor: ->
# Register ports
@inPorts =
in: new noflo.Port()
@outPorts =
out: new noflo.Port()
@inPorts.in.on "data", (data) =>
# Forward data when we receive it.
# Note: send() will connect automatically if needed
@outPorts.out.send data
@inPorts.in.on "disconnect", =>
# Disconnect output port when input port disconnects
@outPorts.out.disconnect()
exports.getComponent = ->
new Forwarder()
This example component register two ports: in and out. When it receives data in the in port, it opens the out port and sends the same data there. When the in connection closes, it will also close the out connection. So basically this component would be a simple repeater.
You can find more examples of components in the components folder shipping with NoFlo.
A NoFlo graph may contain multiple subgraphs, managed by instances of the Graph component. Subgraphs are useful for packaging particular flows to be used as a "new component" by other flows. This allows building more advanced functionality by creating reusable graphs of connected components.
The Graph component loads the graph given to it as a new NoFlo network, and looks for unattached ports in it. It then exposes these ports as its own inports or outports. This way a graph containing subgraphs can easily connect data between the main graph and the subgraph.
Unattached ports from the subgraph will be available through naming ProcessName.port on the Graph component instance.
Simple example, specifying what file a spreadsheet-parsing subgraph should run with:
# Load a subgraph as a new process
'examples/spreadsheet/parse.fbp' -> GRAPH Reader(Graph)
# Send the filename to the component (subgraph)
'somefile.xls' -> READ.SOURCE Reader()
# Display the results
Reader() ENTITIZE.OUT -> IN Display(Output)
Just like with components, it is possible to share subgraphs via NPM. You have to register them in your package.json, for example:
"name": "noflo-spreadsheet",
"noflo": {
"graphs": {
"Parse": "./graphs/parse.fbp"
}
}
After this the subgraph is available as a "virtual component" with the name spreadsheet/Parse and can be used just like any other component. Subgraphs exported in this manner can be in either JSON or the .fbp format.
Components should aim to be reusable, to do one thing and do it well. This is why often it is a good idea to split functionality traditionally done in one function to multiple components. For example, counting lines in a text file could happen in the following way:
This way the whole logic of the application is in the graph, in how the components are wired together. And each of the components is easily reusable for other purposes.
If a component requires configuration, the good approach is to set sensible defaults in the component, and to allow them to be overridden via an input port. This method of configuration allows the settings to be kept in the graph itself, or for example to be read from a file or database, depending on the needs of the application.
The components should not depend on a particular global state, either, but instead attempt to keep the input and output ports their sole interface to the external world. There may be some exceptions, like a component that listens for HTTP requests or Redis pub-sub messages, but even in these cases the server, or subscription should be set up by the component itself.
When discussing how to solve the unnecessary complexity of software, Out of the Tar Pit promotes an approach quite similar to the one discussed here:
The first thing that we’re doing is to advocate separating out all complexity of any kind from the pure logic of the system (which - having nothing to do with either state or control - we’re not really considering part of the complexity).
Done this way, components represent the pure logic, and the control flow and state of the application is managed separately of them in the graph. This separation makes the system a lot simpler.
Being a flow-based programming environment, the main action in NoFlo happens through ports and their connections. There are several events that can be associated with ports:
It depends on the nature of the component how these events may be handled. Most typical components do operations on a whole transmission, meaning that they should wait for the disconnect event on inports before they act, but some components can also act on single data packets coming in.
When a port has no connections, meaning that it was initialized without a connection, or a detach event has happened, it should do no operations regarding that port.
NoFlo comes with a command shell that you can use to load, run and manipulate NoFlo graphs. For example, the line count graph that was explained in Component design could be built with the shell in the following way:
$ noflo
NoFlo>> new countlines
countlines>> add read ReadFile
countlines>> add split SplitStr
countlines>> add count Counter
countlines>> add display Output
countlines>> connect read out split in
countlines>> connect split out count in
countlines>> connect count count display in
countlines>> dot
digraph {
read [shape=box]
split [shape=box]
count [shape=box]
display [shape=box]
read -> split[label='out']
split -> count[label='out']
count -> display[label='count']
}
countlines>> send read source somefile
You can run help to see all available NoFlo shell commands, and quit to get out of the shell.
As of version 2.6 onwards, the DrawFBP GUI tool for designing Flow-Based Programming graphs is able to generate graphs compatible with NoFlo.
The graphs can be exported to NoFlo format from the File -> Generate network -> NoFlo menu and then run normally.
In addition to the shell, NoFlo also comes with a web interface that allows loaded graphs to be monitored. To start it, load a graph into the NoFlo shell, and run:
>> startserver 8080
This will start the NoFlo monitor on port 8080 of your system, so browsers can connect to it with http://localhost:8080. You can also use another port number.
At the moment the monitor only displays the graph, showing the processes and connections between them. Real-time statistics of data flow, and support for visual graph editing are planned.
In addition to using NoFlo in embedded mode where you create the FBP graph programmatically (see example), you can also initialize and run graphs defined using a JSON file.
The NoFlo JSON files declare the processes used in the FBP graph, and the connections between them. They look like the following:
{
"properties": {
"name": "Count lines in a file"
},
"processes": {
"Read File": {
"component": "ReadFile"
},
"Split by Lines": {
"component": "SplitStr"
},
...
},
"connections": [
{
"data": "package.json",
"tgt": {
"process": "Read File",
"port": "source"
}
},
{
"src": {
"process": "Read File",
"port": "out"
},
"tgt": {
"process": "Split by Lines",
"port": "in"
}
},
...
]
}
To run a graph file, you can either use the load command of the NoFlo shell, or do it programmatically:
noflo = require "noflo"
noflo.loadFile "example.json", (network) ->
console.log "Graph loaded"
console.log network.graph.toDOT()
In addition to the JSON format described above, FBP has its own Domain-Specific Language (DSL) for easy graph definition. The syntax is the following:
'somedata' -> PORT Process(Component) sends initial data somedata to port PORT of process Process that runs component ComponentA(Component1) X -> Y B(Component2) sets up a connection between port X of process A that runs component Component1 and port Y of process B that runs component Component2You can connect multiple components and ports together on one line, and separate connection definitions with a newline or a comma (,).
Components only have to be specified the first time you mention a new process. Afterwards, simply append empty parentheses (()) after the process name.
Example:
'somefile.txt' -> SOURCE Read(ReadFile) OUT -> IN Split(SplitStr)
Split() OUT -> IN Count(Counter) COUNT -> IN Display(Output)
Read() ERROR -> IN Display()
NoFlo supports the FBP language fully. You can either load a graph with a string of FBP-language commands with:
fbpData = "<some FBP language connections>"
noflo = require "noflo"
noflo.graph.loadFbp fbpData, (graph) ->
console.log "Graph loaded"
console.log graph.toDOT()
The .fbp file suffix is used for files containing FBP language. This means you can load them also the same way as you load JSON files, using the noflo.loadFile method, or the NoFlo shell. Example:
$ noflo examples/linecount/count.fbp
NoFlo development happens on GitHub. Just fork the main repository, make modifications and send a pull request.
To run the unit tests you need nodeunit. Run the tests with:
$ nodeunit test
or:
$ npm test
Flow-based programming in general, including NoFlo can be discussed on the Flow Based Programming Google group.
There is also an IRC channel #fbp on FreeNode.
0.3.0 (December 19th 2012)
User interface:
NoFlo's web-based user interface has been moved to a separate noflo-ui repository
The noflo shell command now uses STDOUT for debug output (when invoked with --debug) instead of STDERR
noflo command is used with the additional -s switchnoflo command is used with the additional -v switchDOT language output from NoFlo was made more comprehensive
NoFlo graphs can now alias their internal ports to more user-friendly names when used as subgraphs. When aliases are used, the other free ports are not exposed via the Graph component. This works in both FBP and JSON formats:
For FBP format graphs:
EXPORT=INTERNALPROCESS.PORT:EXTERNALPORT
For JSON format graphs:
{
"exports": [
{
"private": "INTERNALPROCESS.PORT",
"public": "EXTERNALPORT"
}
]
}
NoFlo internals:
process.nextTick to ensure possible subgraphs are readydebug flag was removed from NoFlo Network class, and the networks were made EventEmitters for more flexible monitoringisSubgraph method tells whether a Component is a subgraph or a regular code componentgraph portaddX methods of Graph now return the object that was added to the graphstart and end eventsnodeId propertyChanges to core components:
group outport, and original packet to out portmissed port instead of dropping themin port when sending packets outdata port when receiving a disconnect on the in port. Its out port is now an ArrayPort/) and strings for splittingNew core components:
source port to the path received via the destination portregexp port. Non-matching packets are sent to the missed portNew component libraries:
FAQs
Flow-Based Programming environment for JavaScript
The npm package noflo receives a total of 449 weekly downloads. As such, noflo popularity was classified as not popular.
We found that noflo demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.