* major source changes:
* split oisd.nl in basic and full variant
* add swedish regional list
* made archive categories for shallalist and utcapitole selectable
via LuCI
* made all list variants of energized and stevenblack selectable
via LuCI
* removed dns filereset mode
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove 'dshield' and 'sysctl' (discontinued)
* switch 'malwaredomains', 'shallalist' and 'winhelp' to https
* add a second regional list for poland (provided by matx1002)
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add anti_ad blocklist source
* made SafeSearch provider configurable, you can limit
SafeSearch to certain providers
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add regional list source for czech/slovak
* add regional list source for korea
* adapt oisd_nl changes, switch to adb-syntax domains
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove dumb list cache
* start adblock processing after adding/removing
list sources via CLI
* add regional list source for france
Signed-off-by: Dirk Brenken <dev@brenken.org>
* removed 'hphosts' from sources (discontinued)
* fixed a "out of range" bug and another small issue
in the f_dnsup function
* add three new sources: 'anudeep', 'stopforumspam' and 'youtube'
* changed 'list' behaviour, the source file has now
a higher precedence than the archive file (see readme)
* update readme, added missing parameters & more
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fix dependency issue
* fix query timeouts in web frontend (seen with many selected lists),
now the query comes back latest after 30 seconds, to prevent any
timeouts, with all results to this point.
* add missing parameter in readme.md
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new package dependencies: coreultis-sort and
a download util with SSL support
* focus on speed (multicore-support) to handle quite big lists
* include 38 pre-configured blocklist sources in a compressed
json file (/etc/adblock/adblock.sources.gz)
* dynamic SafeSearch support for google, bing, duckduckgo,
yandex, youtube and pixabay (CNAME (bind) & IP (dnsmaq, unbound))
* DNS backend autodetection
* Download Utility autodetection
* Report Interface autodetection
* Easy cron wrapper to set an adblock related auto-timer for
automatic blocklist updates
* raw domain/blocklist support (e.g. for dnscrypt support)
* re-add restrictive Jaillist support
* rework online doc
* Complete LuCI rewrite (migrated to client side JS)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove 'ransomware' blocklist by abbuse.ch (discontinued)
from default adblock config
* fix/switch 'someonewhocares' config to https only
* fix curl download parameters to follow redirects and
suppress needless output
* made the tmp directory of sort operations configurable,
set 'adb_sorttmp' accordingly (only supported by 'coreutils-sort')
Signed-off-by: Dirk Brenken <dev@brenken.org>
* use raw procd interface trigger as last resort, if the
adblock config is not available during startup
* fix selective subdomain whitelisting for dnsmasq
* fix a kresd restart issue with 'DNS File Reset'
* fix a suspend/resume cornercase
* disable the tld compression, if the number of blocked domains
is greater than 'adb_maxtld' (default: 100000)
* made the fw portlist configurable (default '53 853 5353')
* preliminary support for inotify-like autoload features
of dns backends like kresd in future Turris OS. If 'adb_dnsinotify'
is set to 'true', all adblock related restarts and the
'DNS File Reset' will be disabled
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add support for 'DNS File Reset', where the final DNS blockfile
will be purged after DNS backend loading (save storage space).
A small background service will be started to trace/handle
dns backend reloads/restarts
* add support for the 'null' blocking variant in dnsmasq
(via addn-hosts), which may provide better response times
in dnsmasq
* enhance the report & search engine to support
the new blocking variants. Search now includes
backups & black-/whitelist as well
* compressed source list backups are now mandatory (default to '/tmp')
* speed up TLD compression
* E-Mail notification setup is now integrated in UCI/LuCI
* update the LuCI frontend to reflect all changes (separate PR)
* drop preliminary dnscrypt-proxy-support (use dnsmasq instead)
* drop additional 'dnsjail' blocklist support (not used by anyone)
* procd cleanups in init
* various shellcheck cleanups
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
- fix reporting for bogus hostnames with underscores
- no longer accidently overwrite existing 'serversfile' entries in dhcp
config which reference to the adblock jail list
- remove needless 'no_mail' flag
- refined log message regarding tcpdump requirement for reporting
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enhance the query function to search in adblock backups as well,
to get back the set of blocking lists sources for a certain domain
* add "Latest DNS Queries" report to commandline version as well
(already in LuCI)
* made the tld compression (the error handling) more robust,
remove the needless 'adb_forcesrt' option
* removed abandoned 'feodo' list source
* updated readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* report engine supports multiple listening ports, set
'adb_replisten' to a space separated list of ports,
default '53'
* report engine supports multiple interfaces, set 'adb_repiface'
to 'any'
* small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add adblock dns query reporting via tcpdump (see readme for details)
* fix tld compression on low memory systems (< 64 MB)
* fix various small issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add generic blocklist archive support
* add support for blacklist archive from Toulouse 1 University Capitole
* add support for urlhaus RPZ domains by abuse.ch
* small bugfixes & enhancements
luci:
* archive sub-categories (shalla & ut_capitole) are now configurable
via LuCI CBI template
* small bugfixes & enhancements
Signed-off-by: Dirk Brenken <dev@brenken.org>
* major performance boost: add a flexible 'Download Queue' to handle
downloads & list processing in parallel, default queue size is '4',
you can raise this e.g. to '8' or '16' to get it really fast
* replace former 'whitelist mode': the new 'Jail' option
builds an additional 'adb_list.jail' list in parallel
which can be used manually for guest wifi or kidsafe configurations
* regex parser & query function now fully support IDN domains
with non-ASCII characters
* add error handling in tld compression,
to handle OOM conditions better
* adblock.notify sends now html emails,
to get a better look & feel, even on mobile devices
* add czech regional blocklist maintained by turris omnia users
* LuCI: Support new 'Download Queue' & 'Jail' options
* LuCI: fix field width in "Runtime Information" section
Signed-off-by: Dirk Brenken <dev@brenken.org>
* preserve DNS cache after adblock processing,
- 'unbound' and 'named' support this (please check readme)
- 'dnsmasq' now uses the 'servers-file' directive
to minimize the reload disruption,
even though the dns cache will be cleared after SIGHUP
- 'kresd' dns cache is persistent by upstream default, anyway
Turris Omnia devices need a small upstream software change
which is not accepted/implemented yet
* email notification in case of an error or domain count < n
(default 0, check readme)
* removed securemecca from default config (service has been closed)
* new separate functions for hash compare and list/overall count
* add missing package dependencies
* various clean-ups
* update documentation
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new function to set/delete options in external uci config files
* kresd: automated 'rpz_file' handling in /etc/config/resolver
* firewall: automated 'force_dns' handling if you
enable or disable adblock
* support sha256sum (default) and md5sum for blocklist
comparison & conditional dns restarts
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* please note: config file update required!
* add 'whitelist only' mode, block access to all domains
except those explicitly listed in the whitelist file
* rework awk regex for all blocklist sources
* include 'third-party' domains for all regional lists
* change adguard url and refine filter ruleset
* use POSIX character classes
* fix regex for whitelist preparation
* fix corner case parsing issues
* fix enable/disable behavior
* various other small fixes
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add kresd & turris omnia support
* add dnscrypt-proxy support
* change start priority to 30, to fix possible trigger issues on slow
booting hardware
* simplify suspend/resume handling (no longer use a hideout directory)
* enhanced LuCI frontend
* many small changes & improvements
* default config change (please update your config!), adblock is now
disabled by default
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add preliminary kresd dns backend support for turris devices,
see readme (experimental / untested!)
* use tld compression for overall list, too
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add bind support (see readme)
* export all blocked domains in one central file (adb_list.overall)
* prerequisite for proper bind support
* much faster sort operation with less memory consumption
* backups are still handled per source separately,
to be more flexible in adding/removing block list sources
* add additional 'wan6' interface trigger in default configuration
* various small fixes & optimizations
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* cosmetics
frontend:
* "Save & Reply" now distinguish between normal and manual/backup mode
and triggers an appropriate reload or a start action
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add new 'manual mode' to re-use blocklist backups during startup,
get fresh lists only via manual reload or restart action
* additional free memory check during dns restart to prevent OOM errors
* removed palevo tracker from default config,
this tracker has been discontinued
* cosmetics
LuCI frontend (see luci repo):
* add new 'manual mode' under extra options
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add "adb_forcedns" to redirect all dns requests
to local resolver (disabled by default)
* add "adb_forcesrt" to enable overall sort / duplicate removal
on low memory devices with less than 64 MB RAM (disabled by default)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* no longer misuse ubus/procd service object for adblock runtime
information, now save all required information directly
in a JSON file/format (/tmp/adb_runtime.json)
* new 'status' init command to print runtime information
* add a configurable interface trigger timeout for
nested or slow modem/router setups,
set 'adb_triggerdelay' accordingly (default 1 second)
* add support for pure http download utilities like wget-nossl
or uclient-fetch without libustream-ssl (http donwloads only!)
* fix stop action
* fix enabled/disabled action
* fix country code in regional list for china
* LuCI update to reflect all changes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* re-order all regional lists in adblocklist/easylist format
* add polish blocklist
* add chinese blocklist
* lists called now 'reg_<ch|pl|ro|ru>'
* changed interface trigger default in config to 'wan' only
* update / sync LuCI frontend (see luci repo for details)
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* revamp LuCI GUI (see luci repo for details)
* integrated runtime statistics in normal adblock ubus service instance
(see readme)
* simplified boot/init
* no longer use raw interface trigger
* use only network interface trigger, pre-configured for
'wan/wwan/lan' interfaces (see readme)
* fixed a reload issue
* removed switch to disable tld compression (seems to be pretty stable)
* muted awk parser if source list contains incorrect records
* raise dns backend timeout to 20 seconds
* documentation update
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add tld compression,
this new "top level domain compression" removes up to 40 thousand
needless host entries from the block lists and
lowers the memory footprint for the dns backends by 8-10 MByte
* optimize restart behavior in case of an error
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* automatically selects dnsmasq or unbound as dns backend
* add the new 'adguard' source, a combined/quite effective block list
* remove needless dns backend restarts
* optimize adblock restart behavior
* optimize block list processing on inotify enabled filesystems
* better return code checking on block list download
* fix boot function/startup on Chaos Calmer
* fix a bug in blocklist removal function
* add more (optional) debug output
* move backup options to global config
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
Dirk Brenken
Claudius Ellsel
rams
Julius-Paul Jann
Patrick Mulder