A lot of people already use adblocker plugins within their desktop browsers, but what if you are using your (smart) phone, tablet, watch or any other (wlan) gadget!? Getting rid of annoying ads, trackers and other abuse sites (like facebook) is simple: block them with your router. When the dns server on your router receives dns requests, you will sort out queries that ask for the resource records of ad servers and return a simple 'NXDOMAIN'. This is nothing but **N**on-e**X**istent Internet or Intranet domain name, if domain name is unable to resolved using the dns server, a condition called the 'NXDOMAIN' occurred.
A lot of people already use adblocker plugins within their desktop browsers, but what if you are using your (smart) phone, tablet, watch or any other (wlan) gadget!? Getting rid of annoying ads, trackers and other abuse sites (like facebook) is simple: block them with your router. When the DNS server on your router receives DNS requests, you will sort out queries that ask for the resource records of ad servers and return a simple 'NXDOMAIN'. This is nothing but **N**on-e**X**istent Internet or Intranet domain name, if domain name is unable to resolved using the DNS server, a condition called the 'NXDOMAIN' occurred.
## Main Features
## Main Features
* support of the following domain blocklist sources (free for private usage, for commercial use please check their individual licenses):
* support of the following domain blocklist sources (free for private usage, for commercial use please check their individual licenses):
@ -12,7 +12,7 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* => static local blacklist, located by default in '/etc/adblock/adblock.blacklist'
* => static local blacklist, located by default in `/etc/adblock/adblock.blacklist`
* [disconnect](https://disconnect.me)
* [disconnect](https://disconnect.me)
* => numerous updates on the same day, approx. 6.500 entries (enabled by default)
* => numerous updates on the same day, approx. 6.500 entries (enabled by default)
* [dshield](http://dshield.org)
* [dshield](http://dshield.org)
@ -61,29 +61,29 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* => weekly updates, approx. 2.500 entries (enabled by default)
* => weekly updates, approx. 2.500 entries (enabled by default)
* zero-conf like automatic installation & setup, usually no manual changes needed
* zero-conf like automatic installation & setup, usually no manual changes needed
* simple but yet powerful adblock engine: adblock does not use error prone external iptables rulesets, http pixel server instances and things like that
* simple but yet powerful adblock engine: adblock does not use error prone external iptables rulesets, http pixel server instances and things like that
* support four different dns backends: dnsmasq, unbound, named (bind) and kresd
* support two different dns blocking variants: 'nxdomain' (default, supported by all backends), 'null' (supported only by dnsmasq)
* support six different download utilities: uclient-fetch, wget, curl, aria2c, wget-nossl, busybox-wget
* support four different DNS backends: `dnsmasq`, `unbound`, `named` (bind) and `kresd`
* support two different DNS blocking variants: `nxdomain` (default, supported by all backends), `null` (supported only by `dnsmasq`)
* support six different download utilities: `uclient-fetch`, `wget`, `curl`, `aria2c`, `wget-nossl`, `busybox-wget`
* fast downloads & list processing as they are handled in parallel running background jobs (see 'Download Queue')
* fast downloads & list processing as they are handled in parallel running background jobs (see 'Download Queue')
* provide 'http only' mode without installed ssl library for all non-SSL blocklist sources
* provide `http only` mode without installed ssl library for all non-SSL blocklist sources
* support a wide range of router modes, even AP modes are supported
* support a wide range of router modes, even AP modes are supported
* full IPv4 and IPv6 support
* full IPv4 and IPv6 support
* provide top level domain compression ('tld compression'), this feature removes thousands of needless host entries from the blocklist and lowers the memory footprint for the dns backend
* provide top level domain compression (`tld compression`), this feature removes thousands of needless host entries from the blocklist and lowers the memory footprint for the DNS backend
* provide a 'DNS File Reset', where the final DNS blockfile will be purged after DNS backend loading to save storage space
* provide a 'DNS File Reset', where the final DNS blockfile will be purged after DNS backend loading to save storage space
* blocklist source parsing by fast & flexible regex rulesets
* blocklist source parsing by fast & flexible regex rulesets
* overall duplicate removal in central blocklist 'adb_list.overall'
* additional blacklist for manual overrides, located by default in /etc/adblock/adblock.blacklist or in LuCI
* additional whitelist for manual overrides, located by default in /etc/adblock/adblock.whitelist or in LuCI
* quality checks during blocklist update to ensure a reliable dns backend service
* overall duplicate removal in central blocklist `adb_list.overall`
* additional blacklist for manual overrides, located by default in `/etc/adblock/adblock.blacklist` or in LuCI
* additional whitelist for manual overrides, located by default in `/etc/adblock/adblock.whitelist` or in LuCI
* quality checks during blocklist update to ensure a reliable DNS backend service
* minimal status & error logging to syslog, enable debug logging to receive more output
* minimal status & error logging to syslog, enable debug logging to receive more output
* procd based init system support (start/stop/restart/reload/suspend/resume/query/status)
* procd based init system support (`start/stop/restart/reload/suspend/resume/query/status`)
* procd network interface trigger support or classic time based startup
* procd network interface trigger support or classic time based startup
* keep the dns cache intact after adblock processing (currently supported by unbound, named and kresd)
* keep the DNS cache intact after adblock processing (currently supported by unbound, named and kresd)
* suspend & resume adblock actions temporarily without blocklist reloading
* suspend & resume adblock actions temporarily without blocklist reloading
* provide comprehensive runtime information via LuCI or via 'status' init command
* provide a detailed DNS Query Report with dns related information about client requests, top (blocked) domains and more
* provide comprehensive runtime information via LuCI or via `status` init command
* provide a detailed DNS Query Report with DNS related information about client requests, top (blocked) domains and more
* provide a query function to quickly identify blocked (sub-)domains, e.g. for whitelisting. This function is also able to search in adblock backups and black-/whitelist, to get back the set of blocking lists sources for a certain domain
* provide a query function to quickly identify blocked (sub-)domains, e.g. for whitelisting. This function is also able to search in adblock backups and black-/whitelist, to get back the set of blocking lists sources for a certain domain
* option to force dns requests to the local resolver
* option to force DNS requests to the local resolver
* automatic blocklist backup & restore, these backups will be used in case of download errors and during startup
* automatic blocklist backup & restore, these backups will be used in case of download errors and during startup
* send notification E-Mails in case of a processing error or if the overall domain count is ≤ 0
* send notification E-Mails in case of a processing error or if the overall domain count is ≤ 0
* add new adblock sources on your own, see example below
* add new adblock sources on your own, see example below
@ -91,109 +91,110 @@ A lot of people already use adblocker plugins within their desktop browsers, but
## Prerequisites
## Prerequisites
* [OpenWrt](https://openwrt.org), tested with the stable release series (19.07) and with the latest snapshot
* [OpenWrt](https://openwrt.org), tested with the stable release series (19.07) and with the latest snapshot
* a usual setup with an enabled dns backend at minimum - dump AP modes without a working dns backend are _not_ supported
* a usual setup with an enabled DNS backend at minimum - dump AP modes without a working DNS backend are _not_ supported
* a download utility:
* a download utility:
* to support all blocklist sources a full version (with ssl support) of 'wget', 'uclient-fetch' with one of the 'libustream-*' ssl libraries, 'aria2c' or 'curl' is required
* for limited devices with real memory constraints, adblock provides also a 'http only' option and supports wget-nossl and uclient-fetch (without libustream-ssl) as well
* to support all blocklist sources a full version (with ssl support) of `wget`, `uclient-fetch` with one of the `libustream-*` ssl libraries, `aria2c` or `curl` is required
* for limited devices with real memory constraints, adblock provides also a `http only` option and supports `wget-nossl` and `uclient-fetch` (without `libustream-ssl`) as well
* for more configuration options see examples below
* for more configuration options see examples below
* E-Mail notification (optional): for E-Mail notification support you need the additional 'msmtp' package
* DNS Query Report (optional): for this detailed report you need the additional package 'tcpdump' or 'tcpdump-mini'
* E-Mail notification (optional): for E-Mail notification support you need the additional `msmtp` package
* DNS Query Report (optional): for this detailed report you need the additional package `tcpdump` or `tcpdump-mini`
## Installation & Usage
## Installation & Usage
* install 'adblock' (_opkg install adblock_)
* at minimum configure the appropriate dns backend ('dnsmasq' by default), the download utility and enable the adblock service in _/etc/config/adblock_
* control the adblock service manually with _/etc/init.d/adblock_ start/stop/restart/reload/suspend/resume/status or use the LuCI frontend
* install `adblock` (`opkg install adblock`)
* at minimum configure the appropriate DNS backend (`dnsmasq` by default), the download utility and enable the adblock service in `/etc/config/adblock`
* control the adblock service manually with `/etc/init.d/adblock``start/stop/restart/reload/suspend/resume/status` or use the LuCI frontend
## LuCI adblock companion package
## LuCI adblock companion package
* it's strongly recommended to use the LuCI frontend to easily configure all powerful aspects of adblock
* it's strongly recommended to use the LuCI frontend to easily configure all powerful aspects of adblock
* the application is located in LuCI under the `Services` menu
## Tweaks
## Tweaks
* **runtime information:** the adblock status is available via _/etc/init.d/adblock status_ (see example below)
* **debug logging:** for script debugging please set the config option 'adb\_debug' to '1' and check the runtime output with _logread -e "adblock"_
* **runtime information:** the adblock status is available via `/etc/init.d/adblock status` (see example below)
* **debug logging:** for script debugging please set the config option `adb\_debug` to `1` and check the runtime output with `logread -e "adblock"`
* **storage expansion:** to process and store all blocklist sources at once it might be helpful to enlarge your temp directory with a swap partition => see [OpenWrt Wiki](https://openwrt.org/docs/guide-user/storage/fstab) for further details
* **storage expansion:** to process and store all blocklist sources at once it might be helpful to enlarge your temp directory with a swap partition => see [OpenWrt Wiki](https://openwrt.org/docs/guide-user/storage/fstab) for further details
* **coreutils sort:** To speedup adblock processing in particular with many enabled blocklist sources it's recommended to install the additional package 'coreutils-sort'
* **add white- / blacklist entries:** add domain black- or whitelist entries to always-deny or -allow certain (sub) domains, by default both lists are empty and located in _/etc/adblock_. Please add one domain per line - ip addresses, wildcards & regex are _not_ allowed (see example below). You need to refresh your blocklists after changes to these static lists.
* **download queue size:** for further download & list processing performance improvements you can raise the 'adb\_maxqueue' value, e.g. '8' or '16' should be safe
* **coreutils sort:** To speedup adblock processing in particular with many enabled blocklist sources it's recommended to install the additional package `coreutils-sort`
* **add white- / blacklist entries:** add domain black- or whitelist entries to always-deny or -allow certain (sub) domains, by default both lists are empty and located in `/etc/adblock`. Please add one domain per line - ip addresses, wildcards & regex are _not_ allowed (see example below). You need to refresh your blocklists after changes to these static lists.
* **download queue size:** for further download & list processing performance improvements you can raise the `adb\_maxqueue` value, e.g. `8` or `16` should be safe
* **scheduled list updates:** for a scheduled call of the adblock service add an appropriate crontab entry (see example below)
* **scheduled list updates:** for a scheduled call of the adblock service add an appropriate crontab entry (see example below)
* **change startup behaviour:** by default the startup will be triggered by the 'wan' procd interface trigger. Choose 'none' to disable automatic startups, 'timed' to use a classic timeout (default 30 sec.) or select another trigger interface
* **suspend & resume adblocking:** to quickly switch the adblock service 'on' or 'off', simply use _/etc/init.d/adblock [suspend|resume]_
* **change startup behaviour:** by default the startup will be triggered by the `wan` procd interface trigger. Choose `none` to disable automatic startups, `timed` to use a classic timeout (default 30 sec.) or select another trigger interface
* **suspend & resume adblocking:** to quickly switch the adblock service `on` or `off`, simply use `/etc/init.d/adblock [suspend|resume]`
* **domain query:** to query the active blocklist for a certain domain, please use the LuCI frontend or run _/etc/init.d/adblock query `<DOMAIN>`_ (see example below)
* **domain query:** to query the active blocklist for a certain domain, please use the LuCI frontend or run _/etc/init.d/adblock query `<DOMAIN>`_ (see example below)
* **add new list sources:** you could add new blocklist sources on your own via uci config, all you need is a source url and an awk one-liner (see example below)
* **add new list sources:** you could add new blocklist sources on your own via uci config, all you need is a source url and an awk one-liner (see example below)
## Further adblock config options
## Further adblock config options
* usually the pre-configured adblock setup works quite well and no manual overrides are needed
* usually the pre-configured adblock setup works quite well and no manual overrides are needed
* the following options apply to the 'global' config section:
* adb\_enabled => main switch to enable/disable adblock service (default: '0', disabled)
* adb\_dns => select the dns backend for your environment: 'dnsmasq', 'unbound', 'named' or 'kresd' (default: 'dnsmasq')
* adb\_dnsvariant => select the blocking variant: 'nxdomain' (default, supported by all backends), 'null (IPv4)' and 'null (IPv4/IPv6)' both options are only supported by dnsmasq
* adb\_fetchutil => name of the used download utility: 'uclient-fetch', 'wget', 'curl', 'aria2c', 'wget-nossl'. 'busybox' (default: 'uclient-fetch')
* adb\_fetchparm => special config options for the download utility (default: not set)
* adb\_trigger => set the startup trigger to a certain interface, to 'timed' or to 'none' (default: 'wan')
* the following options apply to the 'extra' config section:
* adb\_nice => set the nice level of the adblock process and all sub-processes (int/default: '0', standard priority)
* adb\_forcedns => force dns requests to local resolver (bool/default: '0', disabled)
* adb\_maxqueue => size of the download queue to handle downloads & list processing in parallel (int/default: '8')
* adb\_dnsfilereset => the final DNS blockfile will be purged after DNS backend loading to save storage space (bool/default: 'false', disabled)
* adb\_report => enable the background tcpdump gathering process to provide a detailed DNS Query Report (bool/default: '0', disabled)
* adb\_repdir => target directory for dns related report files generated by tcpdump (default: '/tmp')
* adb\_backupdir => target directory for adblock backups (default: '/tmp')
* adb\_mail => send notification E-Mails in case of a processing errors or if the overall domain count is ≤ 0 (bool/default: '0', disabled)
* adb\_mreceiver => receiver address for adblock notification E-Mails (default: not set)
* the following options could be added via "Additional Field" in LuCI and apply to the 'extra' config section as well:
* adb\_dnsdir => target directory for the generated blocklist 'adb_list.overall' (default: not set, use dns backend default)
* adb\_blacklist => full path to the static blacklist file (default: '/etc/adblock/adblock.blacklist')
* adb\_whitelist => full path to the static whitelist file (default: '/etc/adblock/adblock.whitelist')
* adb\_triggerdelay => additional trigger delay in seconds before adblock processing begins (int/default: '2')
* adb\_maxtld => disable the tld compression, if the number of blocked domains is greater than this value (int/default: '100000')
* adb\_portlist => space separated list of fw ports which should be redirected locally (default: '53 853 5353')
* adb\_dnsinotify => disable adblock triggered restarts and the 'DNS File Reset' for dns backends with autoload features (bool/default: 'false', disabled)
* adb\_dnsflush => flush DNS cache after adblock processing, i.e. enable the old restart behavior (bool/default: '0', disabled)
* adb\_repiface => reporting interface used by tcpdump, set to 'any' for multiple interfaces (default: 'br-lan')
* adb\_replisten => space separated list of reporting port(s) used by tcpdump (default: '53')
* adb\_repchunkcnt => report chunk count used by tcpdump (default: '5')
* adb\_repchunksize => report chunk size used by tcpdump in MB (int/default: '1')
* the following options apply to the `global` config section:
* `adb\_enabled` => main switch to enable/disable adblock service (default: `0`, disabled)
* `adb\_dns` => select the DNS backend for your environment: `dnsmasq`, `unbound`, `named` or `kresd` (default: `dnsmasq`)
* `adb\_dnsvariant` => select the blocking variant: `nxdomain` (default, supported by all backends), `null (IPv4)` and `null (IPv4/IPv6)` both options are only supported by `dnsmasq`
* `adb\_fetchutil` => name of the used download utility: `uclient-fetch`, `wget`, `curl`, `aria2c`, `wget-nossl` or `busybox` (default: `uclient-fetch`)
* `adb\_fetchparm` => special config options for the download utility (default: not set)
* `adb\_trigger` => set the startup trigger to a certain interface, to `timed` or to `none` (default: `wan`)
* the following options apply to the `extra` config section:
* `adb\_nice` => set the nice level of the adblock process and all sub-processes (int/default: `0`, standard priority)
* `adb\_forcedns` => force DNS requests to local resolver (bool/default: `0`, disabled)
* `adb\_maxqueue` => size of the download queue to handle downloads & list processing in parallel (int/default: `8`)
* `adb\_dnsfilereset` => the final DNS blockfile will be purged after DNS backend loading to save storage space (bool/default: `false`, disabled)
* `adb\_report` => enable the background tcpdump gathering process to provide a detailed DNS Query Report (bool/default: `0`, disabled)
* `adb\_repdir` => target directory for DNS related report files generated by tcpdump (default: `/tmp`)
* `adb\_backupdir` => target directory for adblock backups (default: `/tmp`)
* `adb\_mail` => send notification E-Mails in case of a processing errors or if the overall domain count is ≤ 0 (bool/default: `0`, disabled)
* `adb\_mreceiver` => receiver address for adblock notification E-Mails (default: not set)
* the following options could be added via "Additional Field" in LuCI and apply to the `extra` config section as well:
* `adb\_dnsdir` => target directory for the generated blocklist `adb_list.overall` (default: not set, use DNS backend default)
* `adb\_blacklist` => full path to the static blacklist file (default: `/etc/adblock/adblock.blacklist`)
* `adb\_whitelist` => full path to the static whitelist file (default: `/etc/adblock/adblock.whitelist`)
* `adb\_triggerdelay` => additional trigger delay in seconds before adblock processing begins (int/default: `2`)
* `adb\_maxtld` => disable the tld compression, if the number of blocked domains is greater than this value (int/default: `100000`)
* `adb\_portlist` => space separated list of fw ports which should be redirected locally (default: `53 853 5353`)
* `adb\_dnsinotify` => disable adblock triggered restarts and the 'DNS File Reset' for DNS backends with autoload features (bool/default: `false`, disabled)
* `adb\_dnsflush` => flush DNS cache after adblock processing, i.e. enable the old restart behavior (bool/default: `0`, disabled)
* `adb\_repiface` => reporting interface used by tcpdump, set to `any` for multiple interfaces (default: `br-lan`)
* `adb\_replisten` => space separated list of reporting port(s) used by tcpdump (default: `53`)
* `adb\_repchunkcnt` => report chunk count used by tcpdump (default: `5`)
* `adb\_repchunksize` => report chunk size used by tcpdump in MB (int/default: `1`)
Adblock deposits the final blocklist 'adb_list.overall' in '/var/lib/unbound' where unbound can find them in its jail, no further configuration needed.
To preserve the DNS cache after adblock processing you need to install 'unbound-control'.
**change default dns backend to 'named' (bind):**
Adblock deposits the final blocklist `adb_list.overall` in `/var/lib/unbound` where unbound can find them in its jail, no further configuration needed.
To preserve the DNS cache after adblock processing you need to install `unbound-control`.
Adblock deposits the final blocklist 'adb_list.overall' in '/var/lib/bind'.
To preserve the DNS cache after adblock processing you need to install & configure 'bind-rdnc'.
To use the blocklist please modify '/etc/bind/named.conf':
<pre><code>
in the 'options' namespace add:
response-policy { zone "rpz"; };
**change default DNS backend to `named` (bind):**
and at the end of the file add:
Adblock deposits the final blocklist `adb_list.overall` in `/var/lib/bind`.
To preserve the DNS cache after adblock processing you need to install & configure `bind-rdnc`.
To use the blocklist please modify `/etc/bind/named.conf`:
* in the `options` namespace add:
```
response-policy { zone "rpz"; };
```
* and at the end of the file add:
```
zone "rpz" {
zone "rpz" {
type master;
type master;
file "/var/lib/bind/adb_list.overall";
file "/var/lib/bind/adb_list.overall";
allow-query { none; };
allow-query { none; };
allow-transfer { none; };
allow-transfer { none; };
};
};
</code></pre>
**change default dns backend to 'kresd':**
```
**change default DNS backend to `kresd`:**
The knot-resolver (kresd) is only available on Turris Omnia devices.
The knot-resolver (kresd) is only available on Turris Omnia devices.
Adblock deposits the final blocklist 'adb_list.overall' in '/etc/kresd', no further configuration needed.
Adblock deposits the final blocklist `adb_list.overall` in `/etc/kresd`, no further configuration needed.
**enable E-Mail notification via msmtp:**
**enable E-Mail notification via msmtp:**
To use the email notification you have to install & configure the package 'msmtp'.
Modify the file '/etc/msmtprc':
<pre><code>
To use the email notification you have to install & configure the package `msmtp`.
Modify the file `/etc/msmtprc`:
```
[...]
[...]
defaults
defaults
auth on
auth on
@ -208,12 +209,12 @@ port 587
from dev.adblock@gmail.com
from dev.adblock@gmail.com
user dev.adblock
user dev.adblock
password xxx
password xxx
</code></pre>
```
Finally enable E-Mail support and add a valid E-Mail address in LuCI.
Finally enable E-Mail support and add a valid E-Mail address in LuCI.
**receive adblock runtime information:**
**receive adblock runtime information:**
<pre><code>
```
/etc/init.d/adblock status
/etc/init.d/adblock status
::: adblock runtime information
::: adblock runtime information
+ adblock_status : enabled
+ adblock_status : enabled
@ -225,10 +226,10 @@ Finally enable E-Mail support and add a valid E-Mail address in LuCI.
This entry removes the following (sub)domains from the blocklist:
This entry removes the following (sub)domains from the blocklist:
@ -297,12 +298,12 @@ This entry removes the following (sub)domains from the blocklist:
This entry does not remove:
This entry does not remove:
where.com
where.com
www.adwhere.com
www.adwhere.com
</code></pre>
```
**query the active blocklist, the backups and black-/whitelist for a certain (sub-)domain, e.g. for whitelisting:**
**query the active blocklist, the backups and black-/whitelist for a certain (sub-)domain, e.g. for whitelisting:**
The query function checks against the submitted (sub-)domain and recurses automatically to the upper top level domain. For every (sub-)domain it returns the first ten relevant results.
The query function checks against the submitted (sub-)domain and recurses automatically to the upper top level domain. For every (sub-)domain it returns the first ten relevant results.
<pre><code>
```
/etc/init.d/adblock query google.com
/etc/init.d/adblock query google.com
:::
:::
::: results for domain 'google.com' in active blocklist
::: results for domain 'google.com' in active blocklist
@ -338,24 +339,24 @@ The query function checks against the submitted (sub-)domain and recurses automa
Adblock already supports an easylist source, called 'reg_ru'. To add the additional local easylist as a new source, copy the existing config source section and change only
Adblock already supports an easylist source, called 'reg_ru'. To add the additional local easylist as a new source, copy the existing config source section and change only
the source name, the url and the description - that's all!
the source name, the url and the description - that's all!
option adb_src_desc 'focus on romanian ads plus generic easylist additions, weekly updates, approx. 9.400 entries'
option adb_src_desc 'focus on romanian ads plus generic easylist additions, weekly updates, approx. 9.400 entries'
</code></pre>
```
2. a bit harder ...
2. a bit harder ...
To add a really new source with different domain/host format you have to write a suitable awk one-liner on your own, so basic awk skills are needed. As a starting point check the already existing awk rulesets 'adb_src_rset' in the config file, probably you need only small changes for your individual list. Download the desired list and test your new awk string locally. The output result should be a sequential list with one domain/host per line - nothing more. If your awk one-liner works quite well, add a new source section to the adblock config file and test the new source.
To add a really new source with different domain/host format you have to write a suitable awk one-liner on your own, so basic awk skills are needed. As a starting point check the already existing awk rulesets `adb_src_rset` in the config file, probably you need only small changes for your individual list. Download the desired list and test your new awk string locally. The output result should be a sequential list with one domain/host per line - nothing more. If your awk one-liner works quite well, add a new source section to the adblock config file and test the new source.
## Support
## Support
Please join the adblock discussion in this [forum thread](https://forum.openwrt.org/t/adblock-support-thread/507) or contact me by mail <dev@brenken.org>
Please join the adblock discussion in this [forum thread](https://forum.openwrt.org/t/adblock-support-thread/507) or contact me by mail <dev@brenken.org>
Claudius Ellsel
5 years ago