github.com/zevenet/nftlb

nftlb

nftlb stands for nftables load balancer, the next generation linux firewall that will replace iptables is adapted to behave as a complete load balancer and traffic distributor.

nftlb is provided with a JSON API, so you can use your preferred health checker to enable/disable backends or virtual services and automate processes with it.

More info: What is nftlb?

Repository Contents

In this repository is included:

• src/: main source code files.
• include/: include files.
• tests/: automated testbed suite with example configuration files and the script exec_tests.sh to run all of them.
• tests/api: automated testbed suite for API interactionand the script api_tests.sh to run all of them.

Requirements

nftlb depends on:

• linux-kernel: Kernel version 4.19 or higher with nftables modules enabled (iptables, ebtables, etc not required).
• nftables: nftables package with libnftables included and its dependencies (libgmp, libmnl and libnftnl).
• libev: Events library for the web service.
• libjansson: JSON parser for the API.

Installation

To build nftlb, just execute:

root# autoreconf -fi
root# ./configure
root# make

Finally, install it:

root# make install

Also, deb packages are available.

Usage

Check out the command help:

# ./nftlb -h

Here is the list of options:

[ -h | --help ]: Show the command help.
[ -l <LEVEL> | --log <LEVEL> ]: The logs will be shown in the syslog file and with this option you can change the loglevel from 0 to 7 (5 by default).
[ -L <OUTPUT> | --log-output <OUTPUT> ]: Set the daemon logs output (0: syslog - default, 1: stdout, 2: stderr, 3: syslog+stdout, 4: syslog+stderr).
[ -c <FILE> | --config <FILE> ]: Initial configuration file, this argument is optional.
[ -k <KEY> | --key <KEY> ]: The authentication key for the web service can be set by command line, or automatically generated. If it's automatically generated, it'll be shown by command line.
[ -e | --exit ]: This option executes the configuration file into nftables rules and then exit, so the web server won't be available.
[ -d | --daemon ]: Run nftlb as a daemon in background.
[ -6 | --ipv6 ]: Enable IPv6 support for the web service listening port.
[ -H <HOST> | --host <HOST> ]: Set the host for the web service (all interfaces by default).
[ -P <PORT> | --port <PORT> ]: Set the TCP port for the web service (5555 by default).
[ -S | --serial ]: Serialize nft commands.
[ -m <MARK> | --masquerade-mark <MARK> ]: Set masquerade mark in hex (80000000 by default).

Note: In order to use sNAT or dNAT modes, ensure you have activated the ip forwarding option in your system.

Controlling the server behaviour using environment variables

You can also specify a custom server key via the NFTLB_SERVER_KEY environment variable. Zero length keys will be ignored.

root# NFTLB_SERVER_KEY="changeme" nftlb -d

JSON configuration file

The configuration files have the following format:

{
	"farms" : [
		{ <object farm 1> },
		{ <object farm 2> },
		{ ... }
	],
	"addresses" : [
		{ <object address 1> },
		{ <object address 2> },
		{ ... }
	],
	"policies" : [
		{ <object policy 1> },
		{ <object policy 2> },
		{ ... }
	]
}

Where every farm object has the following attributes:

{
	"name" : "<string>",				*Name of the service (required)*
	"family": "<ipv4 | ipv6 | dual>",		*Family of the virtual service (ipv4 by default)*
	"virtual-addr": "<ip address>",			*IP address for the virtual service (OBSOLETE)*
	"virtual-ports": "<port list>",			*Port list separated by commas or ranges separated by a hyphen*
	"source-addr": "<ip address>",			*Source IP address instead of masquerading*
	"mode": "<snat | dnat | dsr | stlsdnat | local>",	*Topology to be implemented (required)*
	"protocol": "<tcp | udp | sctp | all>",		*Protocol to be used by the virtual service (tcp by default)*
	"scheduler": "<weight | rr | hash | symhash>",	*Scheduler to be used (round robin by default)*
	"sched-param": "<srcip | dstip | srcport | dstport | srcmac | dstmac | none>",	*Hash input parameters (none by default)*
	"persistence": "<srcip | dstip | srcport | dstport | srcmac | dstmac | none>",	*Configured stickiness between client and backend (none by default)*
	"persist-ttl": "<number>",	*Stickiness timeout in seconds (60 by default)*
	"helper": "<none | ftp | pptp | sip | snmp | tftp>",	*L7 helper to be used (none by default)*
	"log": "<none | input | forward | output>",	*Enable logging (none by default)*
	"log-prefix": "<string|KNAME|TYPE|FNAME|ANAME>",	*Farm log prefix (default "TYPE-FNAME")*
	"log-rtlimit": "<number>[/<second | minute | hour | day | week >]",			*Security logs rate limit (0/second by default)*
	"mark": "<hexadecimal mark>",			*Set mark mask for the farm (none by default)*
	"priority": "<number>",				*Priority availability for backends > 0 (1 by default)*
	"limits-ttl": "<number>",				*Timeout of banned client due to limit protections (120 seconds by default)*
	"new-rtlimit": "<number>[/<second | minute | hour | day | week >]",				*Number of new connections per service (0/second by default)*
	"new-rtlimit-burst": "<number>",			*Number of burst packets (disabled by default)*
	"new-rtlimit-log-prefix": "<string|KNAME|TYPE|FNAME>",	*Farm new rtlimit log prefix (default "KNAME-TYPE-FNAME")*
	"rst-rtlimit": "<number>[/<second | minute | hour | day | week >]",				*Number of tcp resets allowed (0/second by default)*
	"rst-rtlimit-burst": "<number>",			*Number of burst RST packets (disabled by default)*
	"rst-rtlimit-log-prefix": "<string|KNAME|TYPE|FNAME>",	*Farm reset rtlimit log prefix (default "KNAME-TYPE-FNAME")*
	"est-connlimit": "<number>",				*Number of established connections allowed (disabled by default)*
	"est-connlimit-log-prefix": "<string|KNAME|TYPE|FNAME>",	*Farm established connlimit log prefix (default "KNAME-TYPE-FNAME")*
	"tcp-strict": "<on | off>",				*Option to avoid bogus TCP attacks (disabled by default)*
	"tcp-strict-log-prefix": "<string|KNAME|TYPE|FNAME>",	*Farm TCP strict log prefix (default "KNAME-TYPE-FNAME")*
	"verdict": "<log | drop | accept>",			*Verdict to apply when a limit or blacklist/whitelist matches (log and default verdict per list type by default)*
	"flow-offload": "<on | off>",				*Option to enable flow offload (disabled by default)*
	"intra-connect": "<on | off>",				*Option to enable connectivity from the local machine (disabled by default)*
	"queue": "<number>",				*Number of the queue to send the packets to userspace (disabled by default)*
	"state": "<up | down | off | config_error>",			*Set the status of the virtual service (up by default)*
	"addresses" : [					*List of addresses*
		{<object address 1>},
		{<object address 2>},
		{...}
	],
	"backends" : [					*List of backends*
		{<object backend 1>},
		{<object backend 2>},
		{...}
	],
	"sessions" : [					*List of static sessions. It requires persistence enabled.*
		{
		{<object session 1>},
		{<object session 2>},
		{...}
	],
	"policies" : [					*List of policies*
		{
			"name" : "<policy name>",
		},
		{...}
	]
}

Where every address object has the following attributes:

{
	"name" : "<string>",				*Name of the address (required)*
	"family": "<ipv4 | ipv6 | dual>",		*Family of the address (ipv4 by default)*
	"ip-addr": "<ip address>",			*IP address*
	"ports": "<port list>",				*Port list separated by commas or ranges separated by a hyphen*
	"protocol": "<tcp | udp | sctp | all>",		*Protocol to be used by the address (tcp by default)*
	"verdict": "<log | drop | accept>",			*Verdict to apply when a limit or blacklist/whitelist matches (log and default verdict per list type by default)*
	"log-prefix": "<string|KNAME|TYPE|FNAME|ANAME>",	*Address log prefix (default "TYPE-FNAME")*
	"log-rtlimit": "<number>[/<second | minute | hour | day | week >]",			*Security logs rate limit per second (0/second by default)*
}

Where every backend object has the following attributes:

{
	"name" : "<string>",				*Name of the backend (required)*
	"ip-addr": "<ip address>",			*IP address for the backend (required)*
	"port": "<number>",				*Backend port to redirect the connections*
	"source-addr": "<ip address>",			*Source IP address for a certain backend instead of masquerading or virtual service source address*
	"weight": "<number>",				*Weight of the backend (1 by default)*
	"priority": "<number>",				*Priority availability for the backend > 0 (1 by default)*
	"mark": "<hexadecimal mark>",			*Set mark mask for the backend (none by default)*
	"est-connlimit": "<number>",			*Number of established connections allowed per backend (disabled by default)*
	"est-connlimit-log-prefix": "<string|KNAME|TYPE|FNAME|BNAME>",	*Backend established connections log prefix (default "KNAME-FNAME-BNAME")*
	"state": "<up | down | off | available | config_error>",			*Set the status of the backend (up by default)*
}

Where every session object has the following attributes:

{
	"client" : "<ip address>",			*Client with the same format than persistence configuration*
	"backend": "<backend id>",			*Backend ID to set a stickyness between client and backend*
	"expiration": "<time>"				*Dynamic sessions timeout. Static sessions doesn't include this attribute*
}

Where every policy object has the following attributes:

{
	"name" : "<string>",				*Name of the policy (required)*
	"type": "<blacklist | whitelist>",			*Policy type*
	"family": "<ipv4 | ipv6>",			*Family of the policy (ipv4 by default)*
	"log-prefix": "<string|KNAME|TYPE|FNAME|PNAME>",	*Policy established connections log prefix (default "KNAME-TYPE-PNAME-FNAME")*
	"elements" : [					*List of IPs or networks*
		{
			"data" : "<ip or network>"
		},
		{...}
	]
}

You can find some examples in the tests/ folder.

API examples

Once launched nftlb you can manage it through the API.

Virtual service listing.

curl -H "Key: <MYKEY>" http://<NFTLB IP>:5555/farms

Setup a new virtual service.

curl -H "Key: <MYKEY>" -X POST http://<NFTLB IP>:5555/farms -d "@tests/008_snat_ipv4_all_rr.json"

Add a new backend into a virtual service.

curl -H "Key: <MYKEY>" -X POST http://<NFTLB IP>:5555/farms -d '{"farms" : [ { "name" : "myfarm", "backends" : [ { "name" : "mynewbck", "ip-addr" : "192.168.0.150", "state" : "up" } ] } ] }'

Delete a virtual service.

curl -H "Key: <MYKEY>" -X DELETE http://<NFTLB IP>:5555/farms/lb01

Delete a backend of a virtual service.

curl -H "Key: <MYKEY>" -X DELETE http://<NFTLB IP>:5555/farms/lb01/backends/bck1

Get the static and dynamic sessions.

curl -H "Key: <MYKEY>" -X GET http://<NFTLB IP>:5555/farms/lb01/sessions

Addresses listing.

curl -H "Key: <MYKEY>" http://<NFTLB IP>:5555/addresses

Addresses listing.

curl -H "Key: <MYKEY>" http://<NFTLB IP>:5555/addresses

Create a new address.

curl -H "Key: <MYKEY>" -X POST http://<NFTLB IP>:5555/addresses -d '{ "addresses" : [ { "name" : "myaddr", "family": "ipv4", "ip-addr" : "192.168.0.150", "ports": "8080", "protocol": "tcp" } ] }'

Assign an address to a farm.

curl -H "Key: <MYKEY>" -X POST http://<NFTLB IP>:5555/farms -d '{ "farms" : [ { "name" : "myfarm", "addresses" : [ { "name" : "myaddr" } ] } ] }'

Delete an address.

curl -H "Key: <MYKEY>" -X DELETE http://<NFTLB IP>:5555/addresses/myaddr

Delete an assigned address.

curl -H "Key: <MYKEY>" -X DELETE http://<NFTLB IP>:5555/farms/myfarm/addresses/myaddr

Delete an address.

curl -H "Key: <MYKEY>" -X DELETE http://<NFTLB IP>:5555/addresses/myaddr

How it works

nftlb uses the nftables infrastructure to build virtual services, from user to kernel side. In that regard, the expressions numgen (with its random and inc modes) and hash (say jhash and symhash) allows to distribute traffic among several backends among other properties. More information:

https://wiki.nftables.org/wiki-nftables/index.php/Load_balancing https://www.netfilter.org/projects/nftables/manpage.html

This software creates the required tables named nftlb which are exclusive to the nftlb service and should not be managed manually. Ex: ip nftlb, ip6 nftlb, netdev nftlb...

The integration of nftlb with custom firewall rules are possible creating separated tables and playing with chain priorities. More information:

https://www.zevenet.com/knowledge-base_category/nftlb/

Support

Please refer to the netfilter users mailing list