github.com/britannic/blacklist

UBNT edgeos-dnsmasq-blacklist dnsmasq DNS Blacklisting and Redirection

Follow the conversation @ community.ui.com

Donations and Sponsorship

Please show your thanks by donating to the project using Square Cash or PayPal

We greatly appreciate any and all donations - thank you! Funds go to maintaining development servers and networks.

Note: This is 3rd party software and isn't supported or endorsed by Ubiquiti Networks�

Contents

1. Overview
2. Donate
3. Copyright
4. Licenses
5. Latest Version
6. Change Log
7. Features
8. Compatibility
9. Installation
   1. Using apt-get
   2. Using dpkg
10. Upgrade
11. Reconfigure
12. Removal
13. Frequently Asked Questions
    1. Can I donate to project?
    2. Does the install backup my blacklist configuration before deleting it?
    3. Does update-dnsmasq run automatically?
    4. How do I add or delete sources?
    5. How do I back up my blacklist configuration and restore it later?
    6. How do I configure dnsmasq?
    7. How do I configure local file sources instead of internet based ones?
    8. How do I use standalone or failover mode?
    9. How do I disable/enable dnsmasq blacklisting?
    10. How do I exclude or include a host or a domain?
    11. How do I globally exclude or include hosts or a domains?
    12. How do I use the command line switches?
    13. How do can keep my USG configuration after an upgrade, provision or reboot?
    14. How does whitelisting work?
    15. What is the difference between blocking domains and hosts?
    16. Which blacklist sources are installed by default?
    17. How do I troubleshoot dnsmasq blacklisting?

Overview

EdgeMax dnsmasq DNS blacklisting and redirection is inspired by the users at Ubiquiti UISP Community

[Top]

Copyright

• Copyright � 2021 Helm Rock Consulting

[Top]

Licenses

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

   The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the FreeBSD Project.

[Top]

Latest Version

       

Release 1.2.8 (December 18, 2021)

Change recommended by amoeba00

• Remove extinct source malwaredomains.com

[Top]

Change Log

• See changelog for details.

[Top]

Features

• Adds DNS blacklisting integration to the EdgeRouter configuration
• Generates configuration files used directly by dnsmasq to redirect dns lookups
• Integrated with the EdgeMax OS CLI
• Any FQDN in the blacklist will force dnsmasq to return the configured dns redirect IP address

[Top]

Compatibility

• edgeos-dnsmasq-blacklist has been tested on the EdgeRouter ERLite-3, ERPoe-5, ER-X, ER4, UniFi Security Gateway USG3 and USG4 routers
  • EdgeMAX versions: v1.9.7+hotfix.4-v2.0.9-hotfix.2
  • UniFi versions: 4.4.34.5140624-v4.4.56

[Top]

Installation

• Using apt-get - works for all routers
• Using dpkg - best for disk space constrained routers

[Top]

apt-get Installation - ERLite-3, ERPoe-5, ER-X, ER-X-SFP, ER4, UniFi-Gateway-3 & UniFi-Gateway-4

• Add the blacklist debian package repository using the router's CLI shell

configure
set system package repository blacklist components main
set system package repository blacklist description 'Britannic blacklist debian stretch repository'
set system package repository blacklist distribution stretch
set system package repository blacklist url 'https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public/'
commit;save;exit

• Add the GPG signing key

sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key | sudo apt-key add -

• Update the system repositorities and install edgeos-dnsmasq-blacklist

sudo apt-get update && sudo apt-get install -f edgeos-dnsmasq-blacklist

• You may see reports of unmet dependencies, run this command to fix that:

apt --fix-broken install

[Top]

dpkg Installation - best for disk space constrained routers

EdgeRouter ERLite-3, ERPoe-5, ER4, UniFi-Gateway-3 & UniFi-Gateway-4

curl -L -O https://raw.githubusercontent.com/britannic/blacklist/master/edgeos-dnsmasq-blacklist_1.2.8_mips.deb
sudo dpkg -i edgeos-dnsmasq-blacklist_1.2.8_mips.deb

[Top]

EdgeRouter ER-X & ER-X-SFP

• Ensure the router has enough space, by removing unnecessary files

sudo apt-get clean cache
delete system image

• Now download and install the edgeos-dnsmasq-blacklist package

curl -L -O https://raw.githubusercontent.com/britannic/blacklist/master/edgeos-dnsmasq-blacklist_1.2.8_mipsel.deb
sudo dpkg -i edgeos-dnsmasq-blacklist_1.2.8_mipsel.deb

[Top]

Upgrade

• If the repository is set up and you are using apt-get:

sudo apt-get update && sudo apt-get install -f --only-upgrade edgeos-dnsmasq-blacklist

• Note, if you are using dpkg, it cannot upgrade packages, so follow these instructions and the previous package version will be automatically removed before the new package version is installed

[Top]

Reconfigure

• If the Unifi Security Gateway has been re-provisioned you might need to re-enable the blacklists, in order to do so run:

sudo dpkg-reconfigure edgeos-dnsmasq-blacklist

[Top]

Removal

EdgeMAX - All Platforms

sudo apt-get remove --purge edgeos-dnsmasq-blacklist

[Top]

Frequently Asked Questions

How do I disable/enable dnsmasq blacklisting?

• Use these CLI configure commands:
• Disable:

configure
set service dns forwarding blacklist disabled true
commit;save;exit

• Enable:

configure
set service dns forwarding blacklist disabled false
commit;save;exit

[Top]

Does the install backup my blacklist configuration before deleting it?

• If a blacklist configuration already exists, the install routine will automatically back it up to /config/user-data/blacklist.$(date +'%FT%H%M%S').cmds

[Top]

How do I back up my blacklist configuration and restore it later?

• use the following commands (make a note of the file name) in the shell (not in configure):

export DATE=$(date +'%FT%H%M%S'); echo "Backing up blacklist configuration to: /config/user-data/blacklist.${DATE}.cmds"; show configuration commands | grep blacklist > /config/user-data/blacklist.$(date +'%FT%H%M%S').cmds

• After installing the latest version, you can merge your backed up configuration:

configure
.  /config/user-data/blacklist.[date string].cmds
commit;save;exit

• If you prefer to delete the default configuration and restore your previous configuration, run these commands:

configure
delete service dns forwarding blacklist
.  /config/user-data/blacklist.[date string].cmds
commit;save;exit

[Top]

Which blacklist sources are installed by default?

• Use these CLI shell commands to view the current sources or scan the log for previous downloads:

show configuration commands | match source
grep downloaded /var/log/update-dnsmasq.log

[Top]

How do I configure local file sources instead of internet based ones?

• Use these commands to configure a local file source

set service dns forwarding blacklist hosts source myhosts description 'Blacklist file source'
set service dns forwarding blacklist hosts source myhosts dns-redirect-ip 0.0.0.0
set service dns forwarding blacklist hosts source myhosts file /config/user-data/blist.hosts.src

• File contents example for /config/user-data/blist.hosts.src:

gsmtop.net
click.buzzcity.net
ads.admoda.com
stats.pflexads.com
a.glcdn.co
wwww.adleads.com
ad.madvertise.de
apps.buzzcity.net
ads.mobgold.com
android.bcfads.com
req.appads.com
show.buzzcity.net
api.analytics.omgpop.com
r.edge.inmobicdn.net
www.mmnetwork.mobi
img.ads.huntmad.com
creative1cdn.mobfox.com
admicro2.vcmedia.vn
admicro1.vcmedia.vn

[Top]

How do I use standalone or failover mode?

• The update-dnsmasq binary checks for an active configuration in the following locations:

  1. Uses the EdgeOS/UniFi shell API to get the active blacklist configuration
  2. /config/config.boot (only used if -safe switch is used)
  3. /config/user-data/blacklist.failover.cfg (only used if it has been created and the -safe switch is used)

• If no configuration is found, update-dnsmasq will remove any dnsmasq blocklists it previously created

[Top]

How do I keep my USG configuration after an upgrade, provision or reboot?

• Follow these instructions on how to automatically install edgeos-dnsmasq-blacklist
• Generate and download a config.gateway.json file from your USG following these instructions
• Here's a sample config.gateway.json
• Once the config.gateway.json has been generated, it will need to be uploaded to your UniFi controller per the instructions
• Alternatively follow the instructions for how do I use standalone or failover mode?

[Top]

How do I add or delete sources?

• Using the CLI configure command, to delete domains and hosts sources:

configure
delete service dns forwarding blacklist domains source malc0de
delete service dns forwarding blacklist hosts source yoyo.org
commit;save;exit

• To add a source, first check it can serve a text list and also note the prefix (if any) before the hosts or domains, e.g. http://www.malwaredomainlist.com/ has this format:

#               MalwareDomainList.com Hosts List           #
#   http://www.malwaredomainlist.com/hostslist/hosts.txt   #
#         Last updated: Mon, 04 Dec 17 19:18:42 +0000      #


127.0.0.1  localhost
127.0.0.1  0koryu0.easter.ne.jp
127.0.0.1  109-204-26-16.netconnexion.managedbroadband.co.uk
127.0.0.1  1866809.securefastserver.com

• So the prefix is "127.0.0.1 "
• Here's how to creating the source in the CLI:

configure
set service dns forwarding blacklist hosts source malwaredomainlist description '127.0.0.1 based host and domain list'
set service dns forwarding blacklist hosts source malwaredomainlist prefix '127.0.0.1  '
set service dns forwarding blacklist hosts source malwaredomainlist url 'http://www.malwaredomainlist.com/hostslist/hosts.txt'
commit;save;exit

[Top]

How do I globally exclude or include hosts or a domains?

• Use these example commands to globally include or exclude blacklisted entries:

configure
set service dns forwarding blacklist exclude cdn.visiblemeasures.com
set service dns forwarding blacklist include www.nastywebsites.com
commit;save;exit

[Top]

How do I exclude or include a host or a domain?

• Use these example commands to include or exclude blacklisted entries:

configure
set service dns forwarding blacklist domains exclude visiblemeasures.com
set service dns forwarding blacklist domains include domainsnastywebsites.com
set service dns forwarding blacklist hosts exclude cdn.visiblemeasures.com
set service dns forwarding blacklist hosts include www.nastywebsites.com
commit;save;exit

[Top]

How does whitelisting work?

• dnsmasq whitelists any entries it finds in the configuration files domains and hosts (or servers) that have a hash in place of the IP address (the "#" forces dnsmasq to forward the DNS request to the router's configured nameservers)
• e.g. for servers (hosts)

server=/www.bing.com/#

• e.g. for domains

address=/bing.com/#

[Top]

Does update-dnsmasq run automatically?

• Yes, a scheduled task is created and run daily at midnight with a random start delay is used ensure other routers in the same time zone won't overload the source servers.
• The random start delay window is configured in seconds using this command - this example sets the start delay between 1-10800 seconds (0-3 hours):

set system task-scheduler task update_blacklists executable arguments 10800

• It can be reconfigured using these CLI configuration commands:

set system task-scheduler task update_blacklists executable path /config/scripts/update-dnsmasq-cronjob.sh
set system task-scheduler task update_blacklists executable arguments 10800
set system task-scheduler task update_blacklists interval 1d

• For example, to change the execution interval to every 6 hours, use this command:

set system task-scheduler task update_blacklists interval 6h

• In daily use, no additional interaction with update-dnsmasq is required. By default, cron will run update-dnsmasq at midnight each day to download the blacklist sources and update the dnsmasq configuration files in /etc/dnsmasq.d. dnsmasq will automatically be reloaded after the configuration file update is completed.

[Top]

How do I use the command line switches?

• update-dnsmasq has the following commandline switches available:

/config/scripts/update-dnsmasq -h
  -dir string
        Override dnsmasq directory (default "/etc/dnsmasq.d")
  -f <file>
        <file> # Load a config.boot file
  -h    Display help
  -safe
        Fail over to /config/user-data/blacklist.failover.cfg
  -v    Verbose display
  -version

[Top]

How do I configure dnsmasq?

• dnsmasq may need to be configured to ensure blacklisting works correctly
  • Here is an example using the EdgeOS configuration shell

configure
set service dns forwarding cache-size 2048
set service dns forwarding except-interface [Your WAN i/f]
set service dns forwarding name-server [Your choice of IPv4 Internet Name-Server]
set service dns forwarding name-server [Your choice of IPv4 Internet Name-Server]
set service dns forwarding name-server [Your choice of IPv6 Internet Name-Server]
set service dns forwarding name-server [Your choice of IPv6 Internet Name-Server]
set service dns forwarding options bogus-priv
set service dns forwarding options domain-needed
set service dns forwarding options domain=mydomain.local
set service dns forwarding options enable-ra
set service dns forwarding options expand-hosts
set service dns forwarding options localise-queries
set service dns forwarding options strict-order
set service dns forwarding system
set system name-server 127.0.0.1
set system name-server '::1'
commit; save; exit

[Top]

What is the difference between blocking domains and hosts?

• The difference lies in the order of update-dnsmasq's processing algorithm. Domains are processed first and take precedence over hosts, so that a blacklisted domain will force update-dnsmasq's source parser to exclude subsequent hosts from the same domain. This reduces dnsmasq's list of lookups, since it will automatically redirect hosts for a blacklisted domain.

[Top]

How do I troubleshoot dnsmasq blacklisting?

• View last update log:

  tail /var/log/update-dnsmasq.log

  Total entries extracted = Total sites blocked
  Timestamp = Last time the update ran

• Manually update the dnsmasq configuration files from sources:

  sudo /config/scripts/update-dnsmasq -v

• Check if a domain or hostname is being blocked or manually excluded:

  grep [host.domain.tld] /etc/dnsmasq.d/*

• Or:

  grep [domain.tld] /etc/dnsmasq.d/* 

• Note: If the domain/hostname is manually excluded it will appear in one of the *.whitelisted-*.conf files

• Display installed edgeos-dnsmasq-blacklist version:

  sudo /config/scripts/update-dnsmasq -version

[Top]