Browse Source

Merge pull request #12506 from stangri/master-simple-adblock

simple-adblock: racially-neutral names
lilik-openwrt-22.03
Etienne Champetier 4 years ago
committed by GitHub
parent
commit
24ae70370e
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 141 additions and 128 deletions
  1. +1
    -1
      net/simple-adblock/Makefile
  2. +21
    -20
      net/simple-adblock/files/README.md
  3. +27
    -35
      net/simple-adblock/files/simple-adblock.conf
  4. +5
    -0
      net/simple-adblock/files/simple-adblock.conf.update
  5. +87
    -72
      net/simple-adblock/files/simple-adblock.init

+ 1
- 1
net/simple-adblock/Makefile View File

@ -6,7 +6,7 @@ include $(TOPDIR)/rules.mk
PKG_NAME:=simple-adblock
PKG_VERSION:=1.8.3
PKG_RELEASE:=11
PKG_RELEASE:=13
PKG_MAINTAINER:=Stan Grishin <stangri@melmac.net>
PKG_LICENSE:=GPL-3.0-or-later


+ 21
- 20
net/simple-adblock/files/README.md View File

@ -4,13 +4,13 @@ A simple DNSMASQ/Unbound-based AdBlocking service for OpenWrt/LEDE Project.
## Features
- Super-fast due to the nature of supported block lists and parallel downloading/processing of the blacklists.
- Super-fast due to the nature of supported block-lists and parallel downloading/processing of the block-lists.
- Supports hosts files and domains lists for blocking.
- Everything is configurable from Web UI.
- Allows you to easily add your own domains to whitelist or blacklist.
- Allows you to easily add URLs to your own blocked hosts or domains lists to block/whitelist (just put whitelisted domains one per line in the file you're linking).
- Allows you to easily add your own domains to allow-list or block-list.
- Allows you to easily add URLs to your own blocked hosts or domains lists to allow/block-list (just put allowed domains one per line in the file you're linking).
- Supports multiple modes of AdBlocking implementations with DNSMASQ and Unbound.
- Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 reply or to reject access (depending on settings) for blocked domains.
- Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 reply or to reject access (depending on settings) for blocked domains.
- As some of the default lists are using https, reliably works with either wget/libopenssl, uclient-fetch/libustream-mbedtls or curl.
- Very lightweight and easily hackable, the whole script is just one ```/etc/init.d/simple-adblock``` file.
- Retains the downloaded/sorted AdBlocking list on service stop and reuses it on service start (use ```dl``` command if you want to force re-download of the list).
@ -19,7 +19,7 @@ A simple DNSMASQ/Unbound-based AdBlocking service for OpenWrt/LEDE Project.
- Blocks ads inside browsers with [DNS-over-HTTPS proxy](https://en.wikipedia.org/wiki/DNS_over_HTTPS) built-in, like [Mozilla Firefox](https://support.mozilla.org/en-US/kb/firefox-dns-over-https#w_about-dns-over-https) or [Google Chrome/Chromium](https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html) -- with the ```dnsmasq.ipset``` option.
- Proudly made in Canada, using locally-sourced electrons.
If you want a more robust AdBlocking, supporting free memory detection and complex block lists, supporting IDN, check out ```net/adblock``` on [GitHub](https://github.com/openwrt/packages/tree/master/net/adblock/files)/[jsDelivr](https://cdn.jsdelivr.net/gh/openwrt/packages@master/net/adblock/files/README.md).
If you want a more robust AdBlocking, supporting free memory detection and complex block-lists, supporting IDN, check out ```net/adblock``` on [GitHub](https://github.com/openwrt/packages/tree/master/net/adblock/files)/[jsDelivr](https://cdn.jsdelivr.net/gh/openwrt/packages@master/net/adblock/files/README.md).
## Screenshots (luci-app-simple-adblock)
@ -35,9 +35,9 @@ Configuration - Advanced Configuration
![screenshot](https://cdn.jsdelivr.net/gh/stangri/openwrt_packages@master/screenshots/simple-adblock/screenshot08-config-advanced.png "Configuration - Advanced Configuration")
Whitelist and Blocklist Management
Allowed and Blocked Lists Management
![screenshot](https://cdn.jsdelivr.net/gh/stangri/openwrt_packages@master/screenshots/simple-adblock/screenshot08-lists.png "Whitelist and Blocklist Management")
![screenshot](https://cdn.jsdelivr.net/gh/stangri/openwrt_packages@master/screenshots/simple-adblock/screenshot09-lists.png "Allow-list and Block-list Management")
## Requirements
@ -51,7 +51,7 @@ opkg install uclient-fetch libustream-mbedtls ca-bundle ca-certificates; fi
### Requirements for file:// Scheme
If you want to include some local files as the blocklists or whitelists, you need to install ```curl```:
If you want to include some local files as the allow-lists or block-lists, you need to install ```curl```:
```sh
opkg update; opkg install curl;
@ -76,9 +76,9 @@ For IPv6 support additionally install ```ip6tables-mod-nat``` and ```kmod-ipt-na
opkg update; opkg install ip6tables-mod-nat kmod-ipt-nat6;
```
### Requirements for Faster Blocklist Processing
### Requirements for Faster Block-list Processing
The ```coreutils-sort``` is an optional, but recommended package as it speeds up sorting and removing duplicates from the merged list dramatically. If opkg complains that it can't install ```coreutils-sort``` because /usr/bin/sort is already provided by busybox, you can run the following command:
The ```coreutils-sort``` is an optional, but recommended package as it speeds up sorting and removing duplicates from the merged list dramatically. If opkg complains that it can't install ```coreutils-sort``` because /usr/bin/sort is already provided by busybox, you can run the following command:
```sh
opkg --force-overwrite install coreutils-sort
@ -110,11 +110,11 @@ You can use Web UI (found in Services/Simple AdBlock) to add/remove/edit links t
- [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) (127.0.0.1 or 0.0.0.0 followed by space and domain name per line) to be blocked.
- domains lists (one domain name per line) to be blocked.
- domains lists (one domain name per line) to be whitelisted. It is useful if you want to run ```simple-adblock``` on multiple routers and maintain one centralized whitelist which you can publish on a web-server.
- domains lists (one domain name per line) to be allowed. It is useful if you want to run ```simple-adblock``` on multiple routers and maintain one centralized allow-list which you can publish on a web-server.
Please note that these lists **must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Some of the top block lists (both hosts files and domains lists) suitable for routers with at least 8MB RAM are used in the default ```simple-adblock``` installation.
Please note that these lists **must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Some of the top block-lists (both hosts files and domains lists) suitable for routers with at least 8MB RAM are used in the default ```simple-adblock``` installation.
You can also use Web UI to add individual domains to be blocked or whitelisted.
You can also use Web UI to add individual domains to be blocked or allowed.
If you want to use CLI to customize ```simple-adblock``` config, refer to the [Customization Settings](#customization-settings) section.
@ -146,10 +146,11 @@ In the Web UI the ```simple-adblock``` settings are split into ```basic``` and `
|Advanced|debug|boolean|0|If enabled, output service full debug to ```/tmp/simple-adblock.log```. Please note that the debug file may clog up the router's RAM on some devices. Use with caution.|
|Advanced|allow_non_ascii|boolean|0|Enable support for non-ASCII characters in the final AdBlocking file. Only enable if your target service supports non-ASCII characters. If you enable this on the system where DNS resolver doesn't support non-ASCII characters, it will crash. Use with caution.|
|Advanced|compressed_cache|boolean|0|Create compressed cache of the AdBlocking file in router's persistent memory. Only recommended to be used on routers with large ROM and/or routers with metered/flaky internet connection.|
||whitelist_domain|list/string||List of white-listed domains.|
||whitelist_domains_url|list/string||List of URL(s) to text files containing white-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Useful if you want to keep/publish a single white-list for multiple routers.|
||blacklist_domains_url|list/string||List of URL(s) to text files containing black-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
||blacklist_hosts_url|list/string||List of URL(s) to [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) containing black-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
||allowed_domain|list/string||List of allowed domains.|
||allowed_domains_url|list/string||List of URL(s) to text files containing allowed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Useful if you want to keep/publish a single allow-list for multiple routers.|
||blocked_domain|list/string||List of blocked domains.|
||blocked_domains_url|list/string||List of URL(s) to text files containing blocked domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
||blocked_hosts_url|list/string||List of URL(s) to [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) containing block-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
### DNS Resolution Option
@ -165,11 +166,11 @@ Currently supported options are:
## How Does It Work
This service downloads (and processes in the background, removing comments and other useless data) lists of hosts and domains to be blocked, combines those lists into one big block list, removes duplicates and sorts it and then removes your whitelisted domains from the block list before converting to to DNSMASQ/Unbound-compatible file and restarting DNSMASQ/Unbound if needed. The result of the process is that DNSMASQ/Unbound return NXDOMAIN or 127.0.0.1 (depending on settings) for the blocked domains.
This service downloads (and processes in the background, removing comments and other useless data) lists of hosts and domains to be blocked, combines those lists into one big block-list, removes duplicates and sorts it and then removes your allowed domains from the block-list before converting to to DNSMASQ/Unbound-compatible file and restarting DNSMASQ/Unbound if needed. The result of the process is that DNSMASQ/Unbound return NXDOMAIN or 127.0.0.1 (depending on settings) for the blocked domains.
If you specify ```google.com``` as a domain to be whitelisted, you will have access to ```google.com```, ```www.google.com```, ```analytics.google.com```, but not fake domains like ```email-google.com``` or ```drive.google.com.verify.signin.normandeassociation.com``` for example. If you only want to allow ```www.google.com``` while blocking all other ```google.com``` subdomains, just specify ```www.google.com``` as domain to be whitelisted.
If you specify ```google.com``` as a domain to be allowed, you will have access to ```google.com```, ```www.google.com```, ```analytics.google.com```, but not fake domains like ```email-google.com``` or ```drive.google.com.verify.signin.normandeassociation.com``` for example. If you only want to allow ```www.google.com``` while blocking all other ```google.com``` subdomains, just specify ```www.google.com``` as domain to be allowed.
In general, whatever domain is specified to be whitelisted; it, along with with its subdomains will be whitelisted, but not any fake domains containing it.
In general, whatever domain is specified to be allowed; it, along with with its subdomains will be allowed, but not any fake domains containing it.
## How It Does Not Work


+ 27
- 35
net/simple-adblock/files/simple-adblock.conf View File

@ -11,73 +11,65 @@ config simple-adblock 'config'
option parallel_downloads '1'
option debug '0'
option compressed_cache '0'
list whitelist_domain 'cdn.jsdelivr.net'
list allowed_domain 'cdn.jsdelivr.net'
# Thu Oct 3 17:54:04 PDT 2019
# File size: 4.0K
list blacklist_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt'
list blocked_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt'
# File size: 12.0K
list blacklist_domains_url 'https://ssl.bblck.me/blacklists/domain-list.txt'
list blocked_domains_url 'https://ssl.bblck.me/blacklists/domain-list.txt'
# File size: 44.0K
list blacklist_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt'
list blocked_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt'
# File size: 44.0K
list blacklist_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt'
# File size: 60.0K
# use just one of the www.dshield.org blocklists
list blacklist_domains_url 'https://www.dshield.org/feeds/suspiciousdomains_Medium.txt'
# File size: 64.0K
# use just one of the www.dshield.org blocklists
# list blacklist_domains_url 'https://www.dshield.org/feeds/suspiciousdomains_Low.txt'
list blocked_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt'
# File size: 584.0K
# blocklist too big for most routers
# list blacklist_domains_url 'https://mirror1.malwaredomains.com/files/justdomains'
# block-list too big for most routers
# list blocked_domains_url 'https://mirror1.malwaredomains.com/files/justdomains'
# File size: 16.0K
list blacklist_hosts_url 'https://adaway.org/hosts.txt'
list blocked_hosts_url 'https://adaway.org/hosts.txt'
# File size: 20.0K
list blacklist_hosts_url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list@master/hosts.txt'
list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list@master/hosts.txt'
# File size: 36.0K
list blacklist_hosts_url 'https://www.malwaredomainlist.com/hostslist/hosts.txt'
list blocked_hosts_url 'https://www.malwaredomainlist.com/hostslist/hosts.txt'
# File size: 80.0K
list blacklist_hosts_url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
list blocked_hosts_url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
# File size: 388.0K
# blocklist may be too big for some routers
# blocklist may block some video-streaming content
# list blacklist_hosts_url 'https://cdn.jsdelivr.net/gh/jawz101/MobileAdTrackers@master/hosts'
# block-list may be too big for some routers
# block-list may block some video-streaming content
# list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/jawz101/MobileAdTrackers@master/hosts'
# File size: 424.0K
# blocklist may be too big for some routers
list blacklist_hosts_url 'http://winhelp2002.mvps.org/hosts.txt'
# block-list may be too big for some routers
list blocked_hosts_url 'http://winhelp2002.mvps.org/hosts.txt'
# File size: 432.0K
# blocklist may be too big for some routers
list blacklist_hosts_url 'https://someonewhocares.org/hosts/hosts'
# block-list may be too big for some routers
list blocked_hosts_url 'https://someonewhocares.org/hosts/hosts'
# File size: 624.0K
# blocklist too big for most routers
# list blacklist_hosts_url 'http://sysctl.org/cameleon/hosts'
# block-list too big for most routers
# list blocked_hosts_url 'http://sysctl.org/cameleon/hosts'
# File size: 1.6M
# blocklist too big for most routers
# list blacklist_hosts_url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts@master/hosts'
# block-list too big for most routers
# list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts@master/hosts'
# File size: 3.1M
# blocklist too big for most routers
# list blacklist_hosts_url 'https://hostsfile.mine.nu/Hosts'
# block-list too big for most routers
# list blocked_hosts_url 'https://hostsfile.mine.nu/Hosts'
# File size: 23.0M
# blocklist too big for most routers
# list blacklist_hosts_url 'https://hosts.oisd.nl/'
# block-list too big for most routers
# list blocked_hosts_url 'https://hosts.oisd.nl/'
# site was down on last check
# list blacklist_domains_url 'http://support.it-mate.co.uk/downloads/hosts.txt'
# list blocked_domains_url 'http://support.it-mate.co.uk/downloads/hosts.txt'

+ 5
- 0
net/simple-adblock/files/simple-adblock.conf.update View File

@ -1,3 +1,8 @@
s|whitelist_domains_url|allowed_domains_url|g
s|whitelist_domain|allowed_domain|g
s|blacklist_hosts_url|blocked_hosts_url|g
s|blacklist_domains_url|blocked_domains_url|g
s|blacklist_domain|blocked_domain|g
s|dbl.oisd.nl|hosts.oisd.nl|g
s|raw.githubusercontent.com/StevenBlack/hosts/|cdn.jsdelivr.net/gh/StevenBlack/hosts@|g
s|raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/|cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list@|g


+ 87
- 72
net/simple-adblock/files/simple-adblock.init View File

@ -3,12 +3,16 @@
# shellcheck disable=SC2039,SC1091
PKG_VERSION='dev-test'
export START=94
export USE_PROCD=1
export LC_ALL=C
# shellcheck disable=SC2034
START=94
# shellcheck disable=SC2034
USE_PROCD=1
LC_ALL=C
export EXTRA_COMMANDS='check dl killcache sizes show'
export EXTRA_HELP=' check Checks if specified domain is found in current blacklist
# shellcheck disable=SC2034
EXTRA_COMMANDS='check dl killcache sizes show'
# shellcheck disable=SC2034
EXTRA_HELP=' check Checks if specified domain is found in current block-list
dl Force-downloads all enabled block-list
sizes Displays the file-sizes of enabled block-lists
show Shows the service last-run status'
@ -44,23 +48,12 @@ readonly sharedMemoryError="/dev/shm/$packageName-error"
readonly sharedMemoryOutput="/dev/shm/$packageName-output"
readonly hostsFilter='/localhost/d;/^#/d;/^[^0-9]/d;s/^0\.0\.0\.0.//;s/^127\.0\.0\.1.//;s/[[:space:]]*#.*$//;s/[[:cntrl:]]$//;s/[[:space:]]//g;/[`~!@#\$%\^&\*()=+;:"'\'',<>?/\|[{}]/d;/]/d;/\./!d;/^$/d;/[^[:alnum:]_.-]/d;'
readonly domainsFilter='/^#/d;s/[[:space:]]*#.*$//;s/[[:space:]]*$//;s/[[:cntrl:]]$//;/[[:space:]]/d;/[`~!@#\$%\^&\*()=+;:"'\'',<>?/\|[{}]/d;/]/d;/\./!d;/^$/d;/[^[:alnum:]_.-]/d;'
readonly checkmark='\xe2\x9c\x93'
readonly xmark='\xe2\x9c\x97'
readonly _OK_='\033[0;32m\xe2\x9c\x93\033[0m'
readonly _FAIL_='\033[0;31m\xe2\x9c\x97\033[0m'
readonly __OK__='\033[0;32m[\xe2\x9c\x93]\033[0m'
readonly __FAIL__='\033[0;31m[\xe2\x9c\x97]\033[0m'
readonly _ERROR_='\033[0;31mERROR\033[0m'
readonly messageSuccess='Success'
readonly messageFail='Fail'
readonly messageDownloading='Downloading'
readonly messageReloading='Reloading'
readonly messageRestarting='Restarting'
readonly messageStarting='Starting'
readonly messageForceReloading='Force-Reloading'
readonly messageProcessing='Processing'
readonly messageStopped='Stopped'
getStatusText() {
local _ret
case "$1" in
@ -70,6 +63,7 @@ getStatusText() {
statusRestarting) _ret="Restarting";;
statusForceReloading) _ret="Force Reloading";;
statusDownloading) _ret="Downloading";;
statusProcessing) _ret="Processing";;
statusError) _ret="Error";;
statusWarning) _ret="Warning";;
statusFail) _ret="Fail";;
@ -85,14 +79,14 @@ getErrorText() {
errorSharedMemory) _ret="failed to access shared memory";;
errorSorting) _ret="failed to sort data file";;
errorOptimization) _ret="failed to optimize data file";;
errorWhitelistProcessing) _ret="failed to process whitelist";;
errorAllowListProcessing) _ret="failed to process allow-list";;
errorDataFileFormatting) _ret="failed to format data file";;
errorMovingDataFile) _ret="failed to move data file '${A_TMP}' to '${outputFile}'";;
errorCreatingCompressedCache) _ret="failed to create compressed cache";;
errorRemovingTempFiles) _ret="failed to remove temporary files";;
errorRestoreCompressedCache) _ret="failed to unpack compressed cache";;
errorRestoreCache) _ret="failed to move '$outputCache' to '$outputFile'";;
errorOhSnap) _ret="failed to create blocklist or restart DNS resolver";;
errorOhSnap) _ret="failed to create block-list or restart DNS resolver";;
errorStopping) _ret="failed to stop $serviceName";;
errorDNSReload) _ret="failed to reload/restart DNS resolver";;
errorDownloadingList) _ret="failed to download";;
@ -108,8 +102,8 @@ output_ok() { output 1 "$_OK_"; output 2 "$__OK__\\n"; }
output_okn() { output 1 "$_OK_\\n"; output 2 "$__OK__\\n"; }
output_fail() { output 1 "$_FAIL_"; output 2 "$__FAIL__\\n"; }
output_failn() { output 1 "$_FAIL_\\n"; output 2 "$__FAIL__\\n"; }
# str_replace() { printf "%b" "$1" | sed -e "s/$(printf "%b" "$2")/$(printf "%b" "$3")/g"; }
# str_contains() { test "$1" != "$(str_replace "$1" "$2" '')"; }
str_replace() { printf "%b" "$1" | sed -e "s/$(printf "%b" "$2")/$(printf "%b" "$3")/g"; }
str_contains() { test "$1" != "$(str_replace "$1" "$2" '')"; }
compare_versions() { test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"; }
is_chaos_calmer() { ubus -S call system board | grep -q 'Chaos Calmer'; }
is_ipset_procd() { compare_versions "$(sed -ne 's/^Version: //p' /usr/lib/opkg/info/firewall.control)" "2019-09-18"; }
@ -139,13 +133,33 @@ output() {
fi
}
export serviceEnabled forceDNS parallelDL debug allowIDN compressedCache
export targetDNS bootDelay dlTimeout curlRetry verbosity=1 led dnsInstance
export whitelist_domains blacklist_domains
export whitelist_domains_urls blacklist_domains_urls blacklist_hosts_urls
export wan_if wan_gw wanphysdev dl_command serviceStatus dl_flag
export outputFilter outputFilterIPv6 outputFile outputGzip outputCache ipv6Enabled
export is_ssl_supported
serviceEnabled=1
forceDNS=1
parallelDL=1
debug=0
compressedCache=0
ipv6Enabled=0
bootDelay=120
dlTimeout=20
curlRetry=3
verbosity=2
led=''
targetDNS=dnsmasq.servers
dnsInstance=0
allowed_domains=''
blocked_domains=''
allowed_domains_urls=''
blocked_domains_urls=''
blocked_hosts_urls=''
dl_command=''
dl_flag=''
outputFilter=''
outputFilterIPv6=''
outputFile=''
outputGzip=''
outputCache=''
isSSLSupported=''
allowIDN=0
load_package_config() {
config_load "$packageName"
@ -162,11 +176,11 @@ load_package_config() {
config_get led 'config' 'led'
config_get targetDNS 'config' 'dns' 'dnsmasq.servers'
config_get dnsInstance 'config' 'dns_instance' '0'
config_get whitelist_domains 'config' 'whitelist_domain'
config_get blacklist_domains 'config' 'blacklist_domain'
config_get whitelist_domains_urls 'config' 'whitelist_domains_url'
config_get blacklist_domains_urls 'config' 'blacklist_domains_url'
config_get blacklist_hosts_urls 'config' 'blacklist_hosts_url'
config_get allowed_domains 'config' 'allowed_domain'
config_get allowed_domains_urls 'config' 'allowed_domains_url'
config_get blocked_domains 'config' 'blocked_domain'
config_get blocked_domains_urls 'config' 'blocked_domains_url'
config_get blocked_hosts_urls 'config' 'blocked_hosts_url'
if [ "$targetDNS" != 'dnsmasq.addnhosts' ] && [ "$targetDNS" != 'dnsmasq.conf' ] && \
[ "$targetDNS" != 'dnsmasq.servers' ] && [ "$targetDNS" != 'unbound.adb_list' ] && \
@ -249,13 +263,14 @@ load_package_config() {
|| grep -q "libustream-mbedtls" /usr/lib/opkg/status \
|| grep -q "libustream-openssl" /usr/lib/opkg/status \
|| grep -q "libustream-wolfssl" /usr/lib/opkg/status; then
is_ssl_supported=1
isSSLSupported=1
else
unset is_ssl_supported
unset isSSLSupported
fi
}
is_enabled() {
wan_if=''; wan_gw='';
load_package_config
if [ "$debug" -ne 0 ]; then
@ -457,7 +472,7 @@ tmpfs() {
stats)
printf "%b" "$stats"; return;;
triggers)
curReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
curReload="$parallelDL $debug $dlTimeout $allowed_domains $blocked_domains $allowed_domains_urls $blocked_domains_urls $blocked_hosts_urls $targetDNS"
curRestart="$compressedCache $forceDNS $led"
if [ ! -s "$jsonFile" ]; then
ret='on_boot'
@ -513,7 +528,7 @@ tmpfs() {
stats)
stats="$value";;
triggers)
readReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
readReload="$parallelDL $debug $dlTimeout $allowed_domains $blocked_domains $allowed_domains_urls $blocked_domains_urls $blocked_hosts_urls $targetDNS"
readRestart="$compressedCache $forceDNS $led"
;;
esac
@ -678,7 +693,7 @@ process_url() {
else
type='Allowed'; D_TMP="$A_TMP";
fi
if [ "${1:0:5}" == "https" ] && [ -z "$is_ssl_supported" ]; then
if [ "${1:0:5}" == "https" ] && [ -z "$isSSLSupported" ]; then
output 1 "$_FAIL_"
output 2 "[DL] $type $label $__FAIL__\\n"
echo "errorNoSSLSupport|${1}" >> "$sharedMemoryError"
@ -710,7 +725,7 @@ process_url() {
download_lists() {
local hf w_filter j=0 R_TMP
tmpfs set message "${messageDownloading}..."
tmpfs set message "$(getStatusText "statusDownloading")..."
tmpfs set status "statusDownloading"
rm -f "$A_TMP" "$B_TMP" "$outputFile" "$outputCache" "$outputGzip"
@ -725,8 +740,8 @@ download_lists() {
touch $A_TMP; touch $B_TMP;
output 1 'Downloading lists '
rm -f "$sharedMemoryError"
if [ -n "$blacklist_hosts_urls" ]; then
for hf in ${blacklist_hosts_urls}; do
if [ -n "$blocked_hosts_urls" ]; then
for hf in ${blocked_hosts_urls}; do
if [ "$parallelDL" -gt 0 ]; then
process_url "$hf" 'hosts' 'blocked' &
else
@ -734,8 +749,8 @@ download_lists() {
fi
done
fi
if [ -n "$blacklist_domains_urls" ]; then
for hf in ${blacklist_domains_urls}; do
if [ -n "$blocked_domains_urls" ]; then
for hf in ${blocked_domains_urls}; do
if [ "$parallelDL" -gt 0 ]; then
process_url "$hf" 'domains' 'blocked' &
else
@ -743,8 +758,8 @@ download_lists() {
fi
done
fi
if [ -n "$whitelist_domains_urls" ]; then
for hf in ${whitelist_domains_urls}; do
if [ -n "$allowed_domains_urls" ]; then
for hf in ${allowed_domains_urls}; do
if [ "$parallelDL" -gt 0 ]; then
process_url "$hf" 'domains' 'allowed' &
else
@ -761,16 +776,16 @@ download_lists() {
rm -f "$sharedMemoryError"
fi
[ -n "$blacklist_domains" ] && for hf in ${blacklist_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done
whitelist_domains="${whitelist_domains}
[ -n "$blocked_domains" ] && for hf in ${blocked_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done
allowed_domains="${allowed_domains}
$(cat $A_TMP)"
[ -n "$whitelist_domains" ] && for hf in ${whitelist_domains}; do hf="$(echo "$hf" | sed 's/\./\\./g')"; w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done
[ -n "$allowed_domains" ] && for hf in ${allowed_domains}; do hf="$(echo "$hf" | sed 's/\./\\./g')"; w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done
[ ! -s "$B_TMP" ] && return 1
output 1 'Processing downloads '
output 2 'Sorting combined list '
tmpfs set message "$messageProcessing: sorting combined list"
tmpfs set message "$(getStatusText "statusProcessing"): sorting combined list"
if [ "$allowIDN" -gt 0 ]; then
if sort -u "$B_TMP" > "$A_TMP"; then
output_ok
@ -793,7 +808,7 @@ $(cat $A_TMP)"
[ "$targetDNS" = 'unbound.adb_list' ]; then
# TLD optimization written by Dirk Brenken (dev@brenken.org)
output 2 'Optimizing combined list '
tmpfs set message "$messageProcessing: optimizing combined list"
tmpfs set message "$(getStatusText "statusProcessing"): optimizing combined list"
# sed -E 'G;:t;s/(.*)(\.)(.*)(\n)(.*)/\1\4\5\2\3/;tt;s/(.*)\n(\.)(.*)/\3\2\1/' is actually slower than awk
if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$A_TMP" > "$B_TMP"; then
if sort "$B_TMP" > "$A_TMP"; then
@ -828,17 +843,17 @@ $(cat $A_TMP)"
mv "$A_TMP" "$B_TMP"
fi
output 2 'Whitelisting domains '
tmpfs set message "$messageProcessing: whitelisting domains"
output 2 'Allowing domains '
tmpfs set message "$(getStatusText "statusProcessing"): allowing domains"
if sed -i "$w_filter" "$B_TMP"; then
output_ok
else
output_failn
tmpfs add error "errorWhitelistProcessing"
tmpfs add error "errorAllowListProcessing"
fi
output 2 'Formatting merged file '
tmpfs set message "$messageProcessing: formatting merged file"
tmpfs set message "$(getStatusText "statusProcessing"): formatting merged file"
if [ -z "$outputFilterIPv6" ]; then
if sed "$outputFilter" "$B_TMP" > "$A_TMP"; then
output_ok
@ -863,23 +878,23 @@ $(cat $A_TMP)"
case "$targetDNS" in
dnsmasq.addnhosts)
output 2 'Creating DNSMASQ addnhosts file '
tmpfs set message "$messageProcessing: creating DNSMASQ addnhosts file"
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ addnhosts file"
;;
dnsmasq.conf)
output 2 'Creating DNSMASQ config file '
tmpfs set message "$messageProcessing: creating DNSMASQ config file"
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ config file"
;;
dnsmasq.ipset)
output 2 'Creating DNSMASQ ipset file '
tmpfs set message "$messageProcessing: creating DNSMASQ ipset file"
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ ipset file"
;;
dnsmasq.servers)
output 2 'Creating DNSMASQ servers file '
tmpfs set message "$messageProcessing: creating DNSMASQ servers file"
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ servers file"
;;
unbound.adb_list)
output 2 'Creating Unbound adb_list file '
tmpfs set message "$messageProcessing: creating Unbound adb_list file"
tmpfs set message "$(getStatusText "statusProcessing"): creating Unbound adb_list file"
;;
esac
if mv "$A_TMP" "$outputFile"; then
@ -890,7 +905,7 @@ $(cat $A_TMP)"
fi
if [ "$compressedCache" -gt 0 ]; then
output 2 'Creating compressed cache '
tmpfs set message "$messageProcessing: creating compressed cache"
tmpfs set message "$(getStatusText "statusProcessing"): creating compressed cache"
if cacheOps 'createGzip'; then
output_ok
else
@ -901,7 +916,7 @@ $(cat $A_TMP)"
rm -f "$outputGzip"
fi
output 2 'Removing temporary files '
tmpfs set message "$messageProcessing: removing temporary files"
tmpfs set message "$(getStatusText "statusProcessing"): removing temporary files"
rm -f "/tmp/${packageName}_tmp.*" "$A_TMP" "$B_TMP" "$outputCache" || j=1
if [ $j -eq 0 ]; then
output_ok
@ -1173,7 +1188,7 @@ check() {
local string="$1"
local c="$(grep -c "$string" "$outputFile")"
if [ ! -s "$outputFile" ]; then
echo "No blacklist ('$outputFile') found."
echo "No block-list ('$outputFile') found."
elif [ -z "$string" ]; then
echo "Usage: /etc/init.d/${packageName} check string"
elif [ "$c" -gt 0 ]; then
@ -1195,7 +1210,7 @@ check() {
grep "$string" "$outputFile" | sed 's|^local-zone: "||;s|" static$||;';;
esac
else
echo "The $string is not found in current blacklist ('$outputFile')."
echo "The $string is not found in current block-list ('$outputFile')."
fi
}
@ -1204,39 +1219,39 @@ sizes() {
load_package_config
echo "# $(date)"
for i in $blacklist_domains_urls; do
for i in $blocked_domains_urls; do
[ "${i//melmac}" != "$i" ] && continue
if $dl_command "$i" $dl_flag /tmp/sast 2>/dev/null && [ -s /tmp/sast ]; then
echo "# File size: $(du -sh /tmp/sast | awk '{print $1}')"
if compare_versions "$(du -sk /tmp/sast)" "500"; then
echo "# blocklist too big for most routers"
echo "# block-list too big for most routers"
elif compare_versions "$(du -sk /tmp/sast)" "100"; then
echo "# blocklist may be too big for some routers"
echo "# block-list may be too big for some routers"
fi
rm -rf /tmp/sast
echo " list blacklist_domains_url '$i'"
echo " list blocked_domains_url '$i'"
echo ""
else
echo "# site was down on last check"
echo "# list blacklist_domains_url '$i'"
echo "# list blocked_domains_url '$i'"
echo ""
fi
done
for i in $blacklist_hosts_urls; do
for i in $blocked_hosts_urls; do
if $dl_command "$i" $dl_flag /tmp/sast 2>/dev/null && [ -s /tmp/sast ]; then
echo "# File size: $(du -sh /tmp/sast | awk '{print $1}')"
if compare_versions "$(du -sk /tmp/sast)" "500"; then
echo "# blocklist too big for most routers"
echo "# block-list too big for most routers"
elif compare_versions "$(du -sk /tmp/sast)" "100"; then
echo "# blocklist may be too big for some routers"
echo "# block-list may be too big for some routers"
fi
rm -rf /tmp/sast
echo " list blacklist_hosts_url '$i'"
echo " list blocked_hosts_url '$i'"
echo ""
else
echo "# site was down on last check"
echo "# list blacklist_hosts_url '$i'"
echo "# list blocked_hosts_url '$i'"
echo ""
fi
done


Loading…
Cancel
Save