* more startup tweaks
* re-use f_log function in helper scripts
* small fixes / polish up for forthcoming 19.07 release
Signed-off-by: Dirk Brenken <dev@brenken.org>
* use raw procd interface trigger as last resort, if the
adblock config is not available during startup
* fix selective subdomain whitelisting for dnsmasq
* fix a kresd restart issue with 'DNS File Reset'
* fix a suspend/resume cornercase
* disable the tld compression, if the number of blocked domains
is greater than 'adb_maxtld' (default: 100000)
* made the fw portlist configurable (default '53 853 5353')
* preliminary support for inotify-like autoload features
of dns backends like kresd in future Turris OS. If 'adb_dnsinotify'
is set to 'true', all adblock related restarts and the
'DNS File Reset' will be disabled
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fix a dns restart issue if 'flush dns cache' is set
* fix a suspend/resume issue, the status wasn't properly updated
* fix a long standing query issue
* rework return code handling, mostly for debugging
* various cleanups & cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* background service: no longer miss "signal" events for the
dns backend (to trigger adblock)
* fix a dns backend reload issue during switch between
different blocking modes
* domain query: report found domains only once in
"null" blocking mode with IPv4 & IPv6 list entries
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fix a possible race condition during DNS file reset on slow hardware
* optimize DNS restart behaviour in 'null' blocking mode
* mute useless warnings
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add support for 'DNS File Reset', where the final DNS blockfile
will be purged after DNS backend loading (save storage space).
A small background service will be started to trace/handle
dns backend reloads/restarts
* add support for the 'null' blocking variant in dnsmasq
(via addn-hosts), which may provide better response times
in dnsmasq
* enhance the report & search engine to support
the new blocking variants. Search now includes
backups & black-/whitelist as well
* compressed source list backups are now mandatory (default to '/tmp')
* speed up TLD compression
* E-Mail notification setup is now integrated in UCI/LuCI
* update the LuCI frontend to reflect all changes (separate PR)
* drop preliminary dnscrypt-proxy-support (use dnsmasq instead)
* drop additional 'dnsjail' blocklist support (not used by anyone)
* procd cleanups in init
* various shellcheck cleanups
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
- fix reporting for bogus hostnames with underscores
- no longer accidently overwrite existing 'serversfile' entries in dhcp
config which reference to the adblock jail list
- remove needless 'no_mail' flag
- refined log message regarding tcpdump requirement for reporting
Signed-off-by: Dirk Brenken <dev@brenken.org>
* the DNS Report now displays the hostname, MAC-Address or
client IP (CLI & LuCI)
* Filter the DNS Query result set for a particular domain, client or
time frame (CLI & LuCI)
* remove needless XHR.Poll-Events from Reporting page in LuCI
* remove needless 'force sort' option in LuCI
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enhance the query function to search in adblock backups as well,
to get back the set of blocking lists sources for a certain domain
* add "Latest DNS Queries" report to commandline version as well
(already in LuCI)
* made the tld compression (the error handling) more robust,
remove the needless 'adb_forcesrt' option
* removed abandoned 'feodo' list source
* updated readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* report engine supports multiple listening ports, set
'adb_replisten' to a space separated list of ports,
default '53'
* report engine supports multiple interfaces, set 'adb_repiface'
to 'any'
* small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add adblock dns query reporting via tcpdump (see readme for details)
* fix tld compression on low memory systems (< 64 MB)
* fix various small issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add low priority mode (nice level 10), disabled by default
* enhance 'Force DNS' to redirect ports 53, 853 and 5353
frontend:
* switch to dynamic XHR polling for runtime information and logfile
viewing
* add new 'Refresh' button to reload blocklists
* various cleanups & small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* enhance the whitelist function. Now sub-domains could be whitelisted
(e.g. 'fakenews.facebook.com'), even if the correspondent tld is
blacklisted (e.g. 'facebook.com') - this makes whitelisting
much more flexible and predictable
* rework the domain query function to adapt the whitelist changes
* refine startup error checks/messages
* small fixes
luci:
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add generic blocklist archive support
* add support for blacklist archive from Toulouse 1 University Capitole
* add support for urlhaus RPZ domains by abuse.ch
* small bugfixes & enhancements
luci:
* archive sub-categories (shalla & ut_capitole) are now configurable
via LuCI CBI template
* small bugfixes & enhancements
Signed-off-by: Dirk Brenken <dev@brenken.org>
* major performance boost: add a flexible 'Download Queue' to handle
downloads & list processing in parallel, default queue size is '4',
you can raise this e.g. to '8' or '16' to get it really fast
* replace former 'whitelist mode': the new 'Jail' option
builds an additional 'adb_list.jail' list in parallel
which can be used manually for guest wifi or kidsafe configurations
* regex parser & query function now fully support IDN domains
with non-ASCII characters
* add error handling in tld compression,
to handle OOM conditions better
* adblock.notify sends now html emails,
to get a better look & feel, even on mobile devices
* add czech regional blocklist maintained by turris omnia users
* LuCI: Support new 'Download Queue' & 'Jail' options
* LuCI: fix field width in "Runtime Information" section
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enable code to support Turris Omnia forthcoming upstream change
(new kresd 'keep_cache' option) to preserve kresd DNS cache
* fix a 'status' race condition while the adblock process
is running in parallel
* various small speed improvements
* rework debug output
* refine blacklist handling
* enable the (empty) blacklist source in the default config
* email notification supports mstmp, even without sendmail symlink
* email notification writes minimal status to log (one-liner)
* LuCI: refine logfile search term
* LuCI: Textarea 'autoscroll down' in logfile view
* LuCI: Left-align blocklist source table plus a more compact design
Signed-off-by: Dirk Brenken <dev@brenken.org>
* preserve DNS cache after adblock processing,
- 'unbound' and 'named' support this (please check readme)
- 'dnsmasq' now uses the 'servers-file' directive
to minimize the reload disruption,
even though the dns cache will be cleared after SIGHUP
- 'kresd' dns cache is persistent by upstream default, anyway
Turris Omnia devices need a small upstream software change
which is not accepted/implemented yet
* email notification in case of an error or domain count < n
(default 0, check readme)
* removed securemecca from default config (service has been closed)
* new separate functions for hash compare and list/overall count
* add missing package dependencies
* various clean-ups
* update documentation
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new function to set/delete options in external uci config files
* kresd: automated 'rpz_file' handling in /etc/config/resolver
* firewall: automated 'force_dns' handling if you
enable or disable adblock
* support sha256sum (default) and md5sum for blocklist
comparison & conditional dns restarts
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* please note: config file update required!
* add 'whitelist only' mode, block access to all domains
except those explicitly listed in the whitelist file
* rework awk regex for all blocklist sources
* include 'third-party' domains for all regional lists
* change adguard url and refine filter ruleset
* use POSIX character classes
* fix regex for whitelist preparation
* fix corner case parsing issues
* fix enable/disable behavior
* various other small fixes
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fix startup issues with backends like dnscrypt-proxy or kresd
which does not come up without an existing block list
* fix a small 'chown' issue
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add kresd & turris omnia support
* add dnscrypt-proxy support
* change start priority to 30, to fix possible trigger issues on slow
booting hardware
* simplify suspend/resume handling (no longer use a hideout directory)
* enhanced LuCI frontend
* many small changes & improvements
* default config change (please update your config!), adblock is now
disabled by default
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add preliminary kresd dns backend support for turris devices,
see readme (experimental / untested!)
* use tld compression for overall list, too
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* made DNS restart conditional (compare list hash values),
to prevent needless restarts of the DNS backend
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add bind support (see readme)
* export all blocked domains in one central file (adb_list.overall)
* prerequisite for proper bind support
* much faster sort operation with less memory consumption
* backups are still handled per source separately,
to be more flexible in adding/removing block list sources
* add additional 'wan6' interface trigger in default configuration
* various small fixes & optimizations
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* cosmetics
frontend:
* "Save & Reply" now distinguish between normal and manual/backup mode
and triggers an appropriate reload or a start action
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add new 'manual mode' to re-use blocklist backups during startup,
get fresh lists only via manual reload or restart action
* additional free memory check during dns restart to prevent OOM errors
* removed palevo tracker from default config,
this tracker has been discontinued
* cosmetics
LuCI frontend (see luci repo):
* add new 'manual mode' under extra options
Signed-off-by: Dirk Brenken <dev@brenken.org>
* made wget default parms compatible with older program versions
* shift dns detection routine to simplify dns override,
just set 'adb_dnslist' to force a particular backend priority
(default: 'dnsmasq unbound')
* reduce ubus polling during dns detection
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* various small fixes & optimizations
LuCI frontend (see luci repo):
* Limit Blacklist/Whitelist Online editing to max. 512 KB, approx.
20.000 domains per list
* Automatically refresh the overview page after button onclick event,
e.g. 'Suspend/Resume' or 'Save & Apply'
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add "adb_forcedns" to redirect all dns requests
to local resolver (disabled by default)
* add "adb_forcesrt" to enable overall sort / duplicate removal
on low memory devices with less than 64 MB RAM (disabled by default)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* change start priority to get all interface trigger events, even on
fast hardware
* made default trigger delay more conservative to fix possible start up
issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
* no longer misuse ubus/procd service object for adblock runtime
information, now save all required information directly
in a JSON file/format (/tmp/adb_runtime.json)
* new 'status' init command to print runtime information
* add a configurable interface trigger timeout for
nested or slow modem/router setups,
set 'adb_triggerdelay' accordingly (default 1 second)
* add support for pure http download utilities like wget-nossl
or uclient-fetch without libustream-ssl (http donwloads only!)
* fix stop action
* fix enabled/disabled action
* fix country code in regional list for china
* LuCI update to reflect all changes
Signed-off-by: Dirk Brenken <dev@brenken.org>
Dirk Brenken