NFT complementing tool for blocking/allowing traffic based on domain names, IP or mac adresses.
Note: This is work in Progress
Tool developed to provide an elegant way for defining access/deny lists for Nftables (nft). Access/deny lists can hold:
Configuration data is read from configuration files and loaded to appropriate NFT set. NFT configuration (/etc/nftables.*) and NFT List configuration is separated.
Domain names resolved by NFT List are resolved using CloudFlare 1.1.1.1 DOH (DNS over HTTP(s)) server. This is actually the most secure way to query domain names. Domain names are resolved to IP's from A and AAAA records.
IPs pointed by DNS can change over time, thus NFT List runs periodic checks to keep firewall coherent with an actual DNS state.
NOTE: Usage of this tool requires knowledge regarding Linux NFT firewall framework. For more information check out NFT Quick reference in 10 minutes. Additional NFT guides and an examples can be found in Useful links section at the bottom of this page.
Modern Linux systems use NFT firewall - successor of Iptables. It comes with a new C structure alike configuration format (look: NFT in short š ).
Firewall is a crucial security element, thus its configuration should be kept straightforward.
TODO: move it configuration:
The NFT List uses "available-enabled pattern" (look: Configuration files and directories š š š ) configuration, that is well known from Apache or Nginx. List resource, such as IPs are read from simple 'line-delimited' text files.
The idea is to avoid mixing NFT configuration with resource lists. Instead, keep them separate, similar to good program design where algorithms and data remain distinct and are not intertwined.
Next points describe how to configure NFT, you can also jump straight to Examples.
Nftables provides so-called sets. Set is a set of elements of the same type, type can be one of: ipv4_addr, ipv6_addr, ether_addr and more (Named sets specifications).
The NFT Sets are an 'anchors' where NFT List attaches the list of elements (IP addresses, macs, domain names).
Advantage of NFT over IPtables is 'atomic' operation. The command nft -f <filename> loads configuration 'on a side'. When configfile is successfully loaded NFT switches in an atomic manner to a new configuration. This means there is no single moment when firewall is partially configured.
As NFT List operates after nft command, this might bring security issues, such as access to machine from banned IP (for very short time).
The solution is to cork a sets with 0.0.0.0/0 and ::/128 masks.
See example below:
table inet filtering_machine {
set allow_ip_list {
type ipv4_addr;
flags timeout;
}
set allow_ip6_list {
type ipv6_addr;
flags timeout;
}
set ban_ip_list {
type ipv4_addr;
elements = { 0.0.0.0/0 }
}
set ban_ip6_list {
type ipv6_addr;
elements = { ::/128 }
}
set allow_mac_list {
type ether_addr;
flags timeout;
}
}
The allow lists should be left empty (no elements), for filling in bit later by NFT List. In this case there us no security thread as firewall would not allow packages (empty set) to pass through. You can put in NFT configuration IP of administrative machine if you suspect the scenario where NFT List does not load, to ensure access to machine.
Ban lists are logical opposites to 'allow' lists. To ensure there is no single moments when 'bad' packege can 'sneak under', mask 0.0.0.0/0 and ::/128 should be used to cork all ipv4_addr and ipv6_addr traffic. NFT List will remove that addresses after ban list is loaded.
TODO: remove
Note, timeout flag of NFT set can be used to define elements to expire automatically. It might be a good feature for a long, constantly being updated lists. More information in TODO.
NFT List at each boot chceks if lists are propperly sealed. If a ste is bound to drop or reject, but no '0.0.0.0/0 cork' is added, warning is issued.
...
TODO: move to an end
Simple example can be foundin: example1.1-workstation.nft and NFT List
example1.1-workstation.list.
Once when NFT with this configuration is loaded (note flush ruleset - that purges all previous settings) allowed_hosts is empty.
Therefore, no other host is allowed to let in, access can be open from command line:
nft add element inet my_table allowed_hosts { 172.22.0.2 }
or NFT List can do this for you. Define file such as:
# /etc/nftdef/allowed_hosts.list
# host that are allowed for connecting this box
@set inet my_table allowed_hosts # load bellow addresses into 'allowed_hosts' set
172.22.0.2 # Peer 2
and load:
nftlist.sh update /etc/nftdef/allowed_hosts.list
NFT List comes with OpenRC init script to load settings a boot time, and cron script for periodic reloads. More in Part II: Installation and Configuration.
This part explains an interaction of NFT List with Nftables user-space tools (firewall).
NFT comes with NFT sets that allow on grouping elements of the same type together. Example definition of NFT set looks as follows:
table inet user_defined_table_name {
set user_defined_set_name {
type ipv4_addr;
flags timeout, interval;
auto-merge;
elements = { 10.0.0.0/8 }
}
}
Sets can be anonymous, or named. Above example defines named (user_defined_set_name) set.
To have NFT List to feed sets with a data it must be obviously named.
NFT defines various types of elements, for instance ipv4_addr is for keeping IPv4 addresses, ether_addr for mac addresses while ifname is to group network interfaces (e.g. enp7s0, br0).
NFT List operates on a Sets of the following types:
Nftables also provides typeof keyword that as documentation states:
allows you to use a high level expression, then let nftables resolve the base type for you
NFT List can operate only on sets declared with type. Trial of loading data into typeof,
or type but of not supported type will fail.
Set can be complemented by flags that specifies more precisely behaviour and format of an elements:
192.168.100-192.168.199NFT List validates these flags and acts as follows:
NFT List resolves IP addresses and adds them to 'timeout set' setting up a default timeout that is 3 days. If the list is refreshed, timeout is updated. If given domain name does not resolve to a certain IP anymore, it will simpli expire and eventually disappear from the set.NFT List acts normally if set is not bound with rule.NFT List extends accepted format by network prefixes and address ranges.auto-merge option of NFT Set combines a set elements into intervals, this enables auto-packing addresses into groups. For instance
adding 10.2.0.0/16 and 11.3.0.0/16 merges into one network: 10.2.0.0/15.
With auto-merge defined the result of nftlist update might not only be insertions and removals, but also merge, split actions.
In NFT there is no type inet_addr that would match both types ipv4_addr and ipv6_addr. Therefore, one set can not
hold mixed IPv4 and IPv6 elements. If set has been defined of a type ipv4_addr NFT-List will resolve A DNS records to acquire IP.
If set is of type ipv6_addr AAAA DNS records are queried.
If you inspect an example from Quick introduction there are three noticeable aspects:
Sets are a meeting point. NFT List fills in Set with a desired IP/mac addresses.
NFT List does not create, modify or delete chains, rules or other 'structural' firewall settings.
It only operates within sets, 'NFT List' can only:
It's administrator's job to define a firewall configuration. NFT List is a tool designed for filling and keeping set elements up to date.
This part is about how to configure the tool.
NFT list is in an early development phase, there is no packages developed yet, you can install it by unpacking
release file into / directory:
tar -xzvf nfthelper-1.2.9-alpha.tar.gz -C /
this will simply extract nftlist.sh to /usr/local/bin/ and nftlist to etc/init.d/.
At this point, only OpenRC is supported (no systemd).
Add nftlist to OpenRC's default runlevel:
rc-update add nftlist default
To launch NFT List simply start the service:
service nftlist start
this reads configuration from /etc/nftlists/available/ and loads NFT sets.
Just after installation as configuration is empty nothing will happen.
Refer to your Linux distribution guide to learn how to handle NFT configuration under your system.
Usually (Debian, Alphine) main configuration file is /etc/nftables.nft while directory /etc/nftables.d/ is used for drop in additional settings.
Nft List uses /etc/nftlists/ as a storage for network resource lists.
It holds available and enabled directories:
# tree /etc/nftlists/
/etc/nftlists/
āāā available
ā āāā inbound_filter.list
ā āāā outbound_filter.list
āāā enabled
ā āāā outbound_filter.list -> ../available/outbound_filter.list
āāā included
āāā phising_list.txt.gz
3 directories, 4 files
nftlist by default, if no configuration file or directory is passed, loads all files ended with .list extension that are located
in /etc/nftlists/enabled/ (no recursive search).
The file format expected by NFT List is a text file holding domain/address list, such as:
# /etc/nftlists/avaliable/blacklisted_domain.list
@set inet filter blacklist
bad-domain.xxx.com # don't go there
your-bank.login.space # no comment
# aha ...
specialoffernow.info
Comments are marked by \# symbol.
Resource list should or can be proceeded by a directives:
@set <family>|- <table name>|- <set name>|-
Indicates NFT target set for filling, addresses placed below this directive will be loaded to an indicated set.
The directive must define table family, name, and set name. Minus sign (-) (copy sign) if is used for one or more \@set
parameters instructs to apply previous settings (copy previous parameter) e.g.:
@set ipv6 - -
# set will apply to: set ipv6 filter allow_outbound
@set - filter allow_outbound
... addresses ...
# set will apply to: set ipv6 filter allow_inbound
@set - - allow_inbound
... addresses ...
@query <selector>
Do additional DNS query if domain name is encountered, selector indicates query type:
@query subdomain. - query also subdomain, e.g.:
@query www.
also checks www. subdomain for each encountered domain in a current set
@include <file name>
Include file, file that is to be included must be located inside /etc/nftlists/included.
It does not need to have .list extension, it can be compressed, the compression algorithm
must be indicated by a correct extension: .gz, .zip ...
@onpanic keep|discard
This directive defines an action in the case of panic signal. If such an event happen, probably
white lists should be discarded from sets while black addresses keeped.
Panic signal can be issued with:
nftlist panic
command.
The only required directive is \@set that indicates target set. However,
file's \@set directive can be replaced with a command line --set
(or short -s) option that does the same. In this case target set does
not need to be defined in *.list configuration file. For more options see command line usage.
NFT Sets are updated by NFT List each time when revise is started, or reloaded:
service nftlist start
service nftlist reload
NFT List can also be called directly from command line:
# nftlist
Following updates has been found
Id: action:
1 add 4 IPv4 elements
to 'blacklist' set of 'filter' inet table
proceed with update?
yes / no / details / help or (y /n/d/h) - if yes, update all
or yes Ids/ no / details / help or (y Ids/n/d/h) - if yes Ids, update chosen set
:
Typing yes, or y will proceed with update, update can be limited to a chosen set by passing Id
indicating actions just after yes, e.g. yes 1, yes 2 5.
The prompt can be skipped with:
# Update Sets without prompt
nftlist update
# or shorter
nftlist u
This will proceed with updates without prompting.
By default, configuration from /etc/nftlists/available/ is loaded, but it can be overwritten:
nftlist update /etc/my_list.list
# or
nftlist update /etc/my_list_directory/
# additionally include directory can be indicated:
nftlist update /etc/my_list.list --includedir /etc/incl_lists/
# or shortly:
nftlist update /etc/my_list.list -D /etc/incl_lists/
Set can be indicated as a command line argument:
nftlist u --set inet filter allow
# or use short '-s' flag
nftlist u -s inet filter allow
If it is provided as command line argument it does not need to be declared in .list file.
nftlist comes with panic option that will apply policies defined by \@onpanic directive.
A or AAAA records can change over time, therefore firewall shall be updated periodically.
It is advised to add to daily cron service nftlist reload so configuration shall be keep in
sync with DNS entries.
One of the example of whitelisting is to allow outgoing connections to be only possible with a very limited number of servers. This can be a list of a domain names that are used for system updates. The configuration might look as bellow:
# file: /etc/nftlists/available/repo_devuan.list
# Devuan repositories
deb.devuan.org
deb.debian.org
# file: /etc/nftlists/enabled/
@set inet filter allow_outgoing
@include repo_devuan.list
We can whitelist more resources:
# file: /etc/nftlists/enabled/
@set inet filter allow_outgoing
@include repo_devuan.list
10.2.0.100
example.com
Note that by splitting configuration to a set of files it become more manageable. Also note, that you can mix domain names with IP address.
NFT rule bound to allow_outgoing set would be defined in forward chain. The default policy for that chain might be drop, the
rule might be as follows:
iifname br0 oifname eth0 ip daddr @allow_outgoing tcp dport { 80, 443 } accept
In the case of troubles you run:
nftlist --verbose
And check for any detailed error messages. If it does not help analyze what exactly is under the hood:
# Stop firewall
service nftlist stop
service nftables stop
# checkout defined rules
# it should be empty
nft list ruleset
# start firewall
service nftables stop
# analyze configuration
nft list ruleset
# start nftlist
service nftlist start
# and verify if sets has been updated
nft list ruleset
also you can:
# List all sets
nft list sets
# Add set elements manually
nft add element inet my_table allowed_hosts { 172.22.0.2 }
NFT provides counters, it can give you some information if traffic is passing through a certain firewall rule or hook. In tha case of troubles tools such as tcpdump and WireShark come in handy.
NFT by design provides sophisticated functions such as timeouts, intervals, different data types. Additionally, IPs resolved from DNS might change over time. Domain name lists (especially blacklists) do change also, etc. One of the 'musts' for a well-made software is a capability to reload configuration keeping in mind all these sophisticated functions of NFT. Thus, the program requirements has grow comparing original 'read list -> resolve names -> load firewall' concept.
NFT List can be used in:
HowTo:
Theory:
Administration
Bad addresses databases
Programming
FAQs
NFT complementing tool for blocking/allowing traffic based on domain names, IP or mac adresses.
We found that nftlist demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago.Ā It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
npm now supports Trusted Publishing with OIDC, enabling secure package publishing directly from CI/CD workflows without relying on long-lived tokens.
Research
/Security News
A RubyGems malware campaign used 60 malicious packages posing as automation tools to steal credentials from social media and marketing tool users.
Security News
The CNA Scorecard ranks CVE issuers by data completeness, revealing major gaps in patch info and software identifiers across thousands of vulnerabilities.