Browse Source

Merge pull request #2475 from dibdot/adblock

adblock: release 1.0.0
lilik-openwrt-22.03
Hannu Nyman 9 years ago
parent
commit
720572322e
6 changed files with 128 additions and 105 deletions
  1. +2
    -2
      net/adblock/Makefile
  2. +16
    -16
      net/adblock/files/README.md
  3. +43
    -42
      net/adblock/files/adblock-helper.sh
  4. +53
    -37
      net/adblock/files/adblock-update.sh
  5. +6
    -1
      net/adblock/files/adblock.conf
  6. +8
    -7
      net/adblock/files/adblock.init

+ 2
- 2
net/adblock/Makefile View File

@ -7,7 +7,7 @@
include $(TOPDIR)/rules.mk
PKG_NAME:=adblock
PKG_VERSION:=0.91.0
PKG_VERSION:=1.0.0
PKG_RELEASE:=1
PKG_LICENSE:=GPL-3.0+
PKG_MAINTAINER:=Dirk Brenken <openwrt@brenken.org>
@ -24,7 +24,7 @@ endef
define Package/$(PKG_NAME)/description
Powerful adblock script to block ad/abuse domains.
Currently the script supports 15 domain blacklist sites plus manual black- and whitelist overrides.
Currently the script supports 16 domain blacklist sites plus manual black- and whitelist overrides.
Please see https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md for further information.
endef


+ 16
- 16
net/adblock/files/README.md View File

@ -1,10 +1,7 @@
# adblock script for openwrt
## Description
A lot of people already use adblocker plugins within their desktop browsers,
but what if you are using your (smart) phone, tablet, watch or any other wlan gadget...getting rid of annoying ads, trackers and other abuse sites (like facebook ;-) is simple: block them with your router.
When the dns server on your router receives dns requests, you will sort out queries that ask for the resource records of ad servers and return the local ip address of your router and the internal web server delivers a transparent pixel instead.
A lot of people already use adblocker plugins within their desktop browsers, but what if you are using your (smart) phone, tablet, watch or any other wlan gadget...getting rid of annoying ads, trackers and other abuse sites (like facebook ;-) is simple: block them with your router. When the dns server on your router receives dns requests, you will sort out queries that ask for the resource records of ad servers and return the local ip address of your router and the internal web server delivers a transparent pixel instead.
## Main Features
* support of the following domain blocklist sources (free for private usage, for commercial use please check their individual licenses):
@ -24,12 +21,14 @@ When the dns server on your router receives dns requests, you will sort out quer
* => numerous updates on the same day, approx. 1.800 entries
* [palevotracker](https://palevotracker.abuse.ch)
* => daily updates, approx. 15 entries
* [ruadlist+easylist](https://code.google.com/p/ruadlist)
* [ruadlist/easylist](https://code.google.com/p/ruadlist)
* => weekly updates, approx. 2.000 entries
* [shallalist](http://www.shallalist.de) (categories "adv" "costtraps" "spyware" "tracker" "warez" enabled by default)
* => daily updates, approx. 32.000 entries (a short description of all shallalist categories can be found [online](http://www.shallalist.de/categories.html))
* [spam404](http://www.spam404.com)
* => infrequent updates, approx. 5.000 entries
* [sysctl/cameleon](http://sysctl.org/cameleon)
* => weekly updates, approx. 21.000 entries
* [whocares](http://someonewhocares.org)
* => weekly updates, approx. 12.000 entries
* [winhelp](http://winhelp2002.mvps.org)
@ -48,10 +47,10 @@ When the dns server on your router receives dns requests, you will sort out quer
* quality checks during & after update of adblock lists to ensure a reliable dnsmasq service
* basic adblock statistics via iptables packet counters for each chain
* status & error logging to stdout and syslog
* use of dynamic uhttpd instance as adblock pixel server
* use of dynamic iptables ruleset for adblock related redirects/rejects
* use a dynamic uhttpd instance as an adblock pixel server
* use dynamic iptables rulesets for adblock related redirects/rejects
* openwrt init system support (start/stop/restart/reload)
* hotplug support, adblock start will be triggered by wan 'ifup' event
* hotplug support, the adblock start will be triggered by wan 'ifup' event
* optional: adblock list backup/restore (disabled by default)
## Prerequisites
@ -61,20 +60,21 @@ When the dns server on your router receives dns requests, you will sort out quer
* optional: 'kmod-ipt-nat6' for IPv6 support
* the above dependencies and requirements will be checked during package installation & script runtime
## Installation & Usage
* install the adblock package (*opkg update & opkg install adblock*)
## Designated Driver Installation & Usage
* install the adblock package (*opkg install adblock*)
* start the adblock service with */etc/init.d/adblock start* and check *logread -e "adblock"* for adblock related information
* optional: enable/disable your required adblock list sources in */etc/config/adblock* - 'adaway', 'disconnect' and 'yoyo' are enabled by default
* optional: maintain the adblock service in luci under 'System => Startup'
## LuCI adblock companion package
For easy management of the various blocklist sources and and the adblock options there is also a nice & efficient LuCI frontend available.
Please install the package 'luci-app-adblock'. Then you will find the application in LuCI located under 'Services' menu.
Please install the package 'luci-app-adblock' (*opkg install luci-app-adblock*). Then you will find the application in LuCI located under 'Services' menu.
Thanks to Hannu Nyman for this great adblock LuCI frontend!
## CC installation notes
## Chaos Calmer installation notes
* currently the adblock package is *not* part of the CC package repository
* download the latest adblock package *adblock_x.xx.x-1_all.ipk* from a DD snapshot [package directory](https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/packages)
* download the latest adblock package *adblock_x.xx.x-1_all.ipk* from a development snapshot [package directory](https://downloads.openwrt.org/snapshots/trunk/ar71xx/nand/packages/packages)
* due to server hardware troubles the package directory link above may not work, if so please check the [main openwrt download area](https://downloads.openwrt.org) manually
* manual transfer the package to your router and install the opkg package as usual
## Tweaks
@ -83,6 +83,7 @@ Thanks to Hannu Nyman for this great adblock LuCI frontend!
* add personal domain white- or blacklist entries as an additional blocklist source, one domain per line (wildcards & regex are not allowed!), by default both empty lists are located in */etc/adblock*
* enable the backup/restore feature, to restore automatically the latest stable backup of your adblock lists in case of any (partial) processing error (i.e. a single blocklist source server is down). Please use an (external) solid partition and *not* your volatile router temp directory for this
* for a scheduled call of the adblock service via */etc/init.d/adblock start* add an appropriate crontab entry
* in case of any script runtime errors, you should enable script debugging: for this please change the value of the main 'DEBUG' switch, you'll find it in the header of */usr/bin/adblock-update.sh*
## Further adblock config options
* usually the adblock autodetection works quite well and no manual config overrides are needed, all options apply to 'global' adblock config section:
@ -100,11 +101,10 @@ This adblock package is a dns/dnsmasq based adblock solution for openwrt.
Queries to ad/abuse domains are never forwarded and always replied with a local IP address which may be IPv4 or IPv6.
For that purpose adblock uses an ip address from the private 'TEST-NET-1' subnet (192.0.2.1 / ::ffff:c000:0201) by default.
Furthermore all ad/abuse queries will be filtered by ip(6)tables and redirected to internal adblock pixel server (in PREROUTING chain) or rejected (in FORWARD or OUTPUT chain).
All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other openwrt related config files.
There is *no* adblock background daemon running, the (scheduled) start of the adblock service keeps only the adblock lists up-to-date.
All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other openwrt related config files. There is *no* adblock background daemon running, the (scheduled) start of the adblock service keeps only the adblock lists up-to-date.
## Support
Please join the adblock discussion in this [openwrt forum thread](https://forum.openwrt.org/viewtopic.php?id=59803) or contact me by mail <openwrt@brenken.org>
Please join the adblock discussion in this [openwrt forum thread](https://forum.openwrt.org/viewtopic.php?id=59803) or contact me by mail <openwrt@brenken.org>
## Removal
* stop all adblock related services with */etc/init.d/adblock stop*


+ 43
- 42
net/adblock/files/adblock-helper.sh View File

@ -11,12 +11,12 @@ f_envload()
# get version string from default adblock configuration file
#
cfg_version="$(/sbin/uci -q get adblock.global.adb_cfgver 2>/dev/null)"
cfg_enabled="$(/sbin/uci -q get adblock.global.adb_enabled 2>/dev/null)"
cfg_version="$(/sbin/uci -q get adblock.global.adb_cfgver)"
cfg_enabled="$(/sbin/uci -q get adblock.global.adb_enabled)"
rc=$?
if [ $((rc)) -ne 0 ] || [ "${cfg_version}" != "${adb_scriptver%.*}" ]
then
cp -pf "/etc/adblock/adblock.conf.default" "/etc/config/adblock" >/dev/null 2>&1
cp -pf "/etc/adblock/adblock.conf.default" "/etc/config/adblock"
rc=$?
if [ $((rc)) -eq 0 ]
then
@ -36,7 +36,7 @@ f_envload()
#
if [ -r "/lib/functions.sh" ]
then
. "/lib/functions.sh" 2>/dev/null
. "/lib/functions.sh"
else
rc=110
f_log "openwrt function library not found" "${rc}"
@ -47,7 +47,7 @@ f_envload()
#
if [ -r "/lib/functions/network.sh" ]
then
. "/lib/functions/network.sh" 2>/dev/null
. "/lib/functions/network.sh"
else
rc=115
f_log "openwrt network library not found" "${rc}"
@ -62,7 +62,7 @@ f_envload()
f_log "adblock installation finished, 'opkg' currently locked by package installer"
f_exit
fi
pkg_list="$(opkg list-installed 2>/dev/null)"
pkg_list="$(opkg list-installed)"
if [ -z "${pkg_list}" ]
then
rc=120
@ -154,8 +154,8 @@ f_envparse()
#
adb_count=0
adb_minspace=12000
adb_tmpfile="$(mktemp -tu 2>/dev/null)"
adb_tmpdir="$(mktemp -p /tmp -d 2>/dev/null)"
adb_tmpfile="$(mktemp -tu)"
adb_tmpdir="$(mktemp -p /tmp -d)"
adb_dnsdir="/tmp/dnsmasq.d"
adb_dnsprefix="adb_list"
adb_prechain_ipv4="prerouting_rule"
@ -170,27 +170,28 @@ f_envparse()
# set adblock source ruleset definitions
#
rset_core="([A-Za-z0-9_-]+\.){1,}[A-Za-z]+"
rset_adaway="awk '\$0 ~/^127\.0\.0\.1[ ]+${rset_core}/{print tolower(\$2)}'"
rset_adaway="awk '\$0 ~/^127\.0\.0\.1[ \t]+${rset_core}/{print tolower(\$2)}'"
rset_blacklist="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_disconnect="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_dshield="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_feodo="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_malware="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_malwarelist="awk '\$0 ~/^127\.0\.0\.1[ ]+${rset_core}/{print tolower(\$2)}'"
rset_malwarelist="awk '\$0 ~/^127\.0\.0\.1[ \t]+${rset_core}/{print tolower(\$2)}'"
rset_openphish="awk -F '/' '\$3 ~/^${rset_core}/{print tolower(\$3)}'"
rset_palevo="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_ruadlist="awk -F '[|^]' '\$0 ~/^\|\|${rset_core}\^$/{print tolower(\$3)}'"
rset_shalla="awk -F '/' '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_spam404="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_whocares="awk '\$0 ~/^127\.0\.0\.1[ ]+${rset_core}/{print tolower(\$2)}'"
rset_winhelp="awk '\$0 ~/^0\.0\.0\.0[ ]+${rset_core}/{print tolower(\$2)}'"
rset_sysctl="awk '\$0 ~/^127\.0\.0\.1[ \t]+${rset_core}/{print tolower(\$2)}'"
rset_whocares="awk '\$0 ~/^127\.0\.0\.1[ \t]+${rset_core}/{print tolower(\$2)}'"
rset_winhelp="awk '\$0 ~/^0\.0\.0\.0[ \t]+${rset_core}/{print tolower(\$2)}'"
rset_yoyo="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
rset_zeus="awk '\$1 ~/^${rset_core}/{print tolower(\$1)}'"
# get logical wan update interfaces (with default route) and their device names
#
network_find_wan adb_wanif4 2>/dev/null
network_find_wan6 adb_wanif6 2>/dev/null
network_find_wan adb_wanif4
network_find_wan6 adb_wanif6
if [ -z "${adb_wanif4}" ] && [ -z "${adb_wanif6}" ]
then
rc=125
@ -202,15 +203,15 @@ f_envparse()
f_log "LAN only (${adb_lanif}) network, no valid IPv4/IPv6 wan update interface found" "${rc}"
f_exit
else
network_get_device adb_wandev4 "${adb_wanif4}" 2>/dev/null
network_get_device adb_wandev6 "${adb_wanif6}" 2>/dev/null
network_get_device adb_wandev4 "${adb_wanif4}"
network_get_device adb_wandev6 "${adb_wanif6}"
break
fi
# get lan ip addresses
#
network_get_ipaddr adb_ipv4 "${adb_lanif}" 2>/dev/null
network_get_ipaddr6 adb_ipv6 "${adb_lanif}" 2>/dev/null
network_get_ipaddr adb_ipv4 "${adb_lanif}"
network_get_ipaddr6 adb_ipv6 "${adb_lanif}"
if [ -z "${adb_ipv4}" ] && [ -z "${adb_ipv6}" ]
then
rc=130
@ -237,13 +238,13 @@ f_envcheck()
#
if [ -n "${adb_wanif6}" ]
then
check="$(printf "${pkg_list}" | grep "^ip6tables -" 2>/dev/null)"
check="$(printf "${pkg_list}" | grep "^ip6tables -")"
if [ -z "${check}" ]
then
f_log "package 'ip6tables' not found, IPv6 support will be disabled"
unset adb_wanif6
else
check="$(printf "${pkg_list}" | grep "^kmod-ipt-nat6 -" 2>/dev/null)"
check="$(printf "${pkg_list}" | grep "^kmod-ipt-nat6 -")"
if [ -z "${check}" ]
then
f_log "package 'kmod-ipt-nat6' not found, IPv6 support will be disabled"
@ -255,7 +256,7 @@ f_envcheck()
# check ca-certificates package and set wget parms accordingly
#
wget_parm="--no-config --quiet --tries=1 --no-cache --no-cookies --max-redirect=0 --dns-timeout=5 --connect-timeout=5 --read-timeout=5"
check="$(printf "${pkg_list}" | grep "^ca-certificates -" 2>/dev/null)"
check="$(printf "${pkg_list}" | grep "^ca-certificates -")"
if [ -z "${check}" ]
then
wget_parm="${wget_parm} --no-check-certificate"
@ -296,9 +297,9 @@ f_envcheck()
# memory check
#
mem_total="$(awk '$1 ~ /^MemTotal/ {printf $2}' "/proc/meminfo" 2>/dev/null)"
mem_free="$(awk '$1 ~ /^MemFree/ {printf $2}' "/proc/meminfo" 2>/dev/null)"
mem_swap="$(awk '$1 ~ /^SwapTotal/ {printf $2}' "/proc/meminfo" 2>/dev/null)"
mem_total="$(awk '$1 ~ /^MemTotal/ {printf $2}' "/proc/meminfo")"
mem_free="$(awk '$1 ~ /^MemFree/ {printf $2}' "/proc/meminfo")"
mem_swap="$(awk '$1 ~ /^SwapTotal/ {printf $2}' "/proc/meminfo")"
if [ $((mem_total)) -le 64000 ] && [ $((mem_swap)) -eq 0 ]
then
mem_ok="false"
@ -363,20 +364,20 @@ f_envcheck()
# check volatile adblock uhttpd instance configuration
#
rc="$(ps | grep "[u]httpd.*\-h /www/adblock" >/dev/null 2>&1; printf ${?})"
rc="$(ps | grep -q "[u]httpd.*\-h /www/adblock"; printf ${?})"
if [ $((rc)) -ne 0 ]
then
if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ]
then
uhttpd -h "/www/adblock" -k 5 -N 200 -t 0 -T 1 -D -S -E "/index.html" -p "${adb_ipv4}:${adb_port}" -p "[${adb_ipv6}]:${adb_port}">/dev/null 2>&1
uhttpd -h "/www/adblock" -k 5 -N 200 -t 0 -T 1 -D -S -E "/index.html" -p "${adb_ipv4}:${adb_port}" -p "[${adb_ipv6}]:${adb_port}"
rc=${?}
elif [ -n "${adb_wanif4}" ]
then
uhttpd -h "/www/adblock" -k 5 -N 200 -t 0 -T 1 -D -S -E "/index.html" -p "${adb_ipv4}:${adb_port}" >/dev/null 2>&1
uhttpd -h "/www/adblock" -k 5 -N 200 -t 0 -T 1 -D -S -E "/index.html" -p "${adb_ipv4}:${adb_port}"
rc=${?}
elif [ -n "${adb_wanif6}" ]
then
uhttpd -h "/www/adblock" -k 5 -N 200 -t 0 -T 1 -D -S -E "/index.html" -p "[${adb_ipv6}]:${adb_port}" >/dev/null 2>&1
uhttpd -h "/www/adblock" -k 5 -N 200 -t 0 -T 1 -D -S -E "/index.html" -p "[${adb_ipv6}]:${adb_port}"
rc=${?}
fi
if [ $((rc)) -eq 0 ]
@ -414,7 +415,7 @@ f_depend()
local check
local package="${1}"
check="$(printf "${pkg_list}" | grep "^${package} -" 2>/dev/null)"
check="$(printf "${pkg_list}" | grep "^${package} -")"
if [ -z "${check}" ]
then
rc=140
@ -449,10 +450,10 @@ f_firewall()
# check whether iptables rule already applied and proceed accordingly
#
rc="$("${ipt}" -w -t "${table}" -C "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1; printf ${?})"
rc="$("${ipt}" -w -t "${table}" -C "${chain}" -m comment --comment "${notes}" ${rules}; printf ${?})"
if [ $((rc)) -ne 0 ]
then
"${ipt}" -w -t "${table}" -"${ctype}" "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1
"${ipt}" -w -t "${table}" -"${ctype}" "${chain}" -m comment --comment "${notes}" ${rules}
rc=${?}
if [ $((rc)) -eq 0 ]
then
@ -491,7 +492,7 @@ f_log()
log_rc=", rc: ${log_rc}"
log_msg="${log_msg}${log_rc}"
fi
/usr/bin/logger ${log_parm} -t "adblock[${adb_pid}] ${class}" "${log_msg}"
/usr/bin/logger ${log_parm} -t "adblock[${adb_pid}] ${class}" "${log_msg}" 2>&1
fi
}
@ -506,7 +507,7 @@ f_space()
#
if [ -d "${mp}" ]
then
av_space="$(df "${mp}" 2>/dev/null | tail -n1 2>/dev/null | awk '{printf $4}')"
av_space="$(df "${mp}" | tail -n1 | awk '{printf $4}')"
if [ $((av_space)) -lt $((adb_minspace)) ]
then
space_ok="false"
@ -526,7 +527,7 @@ f_restore()
#
if [ -n "${adb_revsrclist}" ]
then
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrclist} \) -print -exec rm -f "{}" \; 2>/dev/null)"
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrclist} \) -print -exec rm -f "{}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${rm_done}" ]
then
@ -542,7 +543,7 @@ f_restore()
#
if [ "${backup_ok}" = "true" ] && [ "$(printf "${adb_backupdir}/${adb_dnsprefix}."*)" != "${adb_backupdir}/${adb_dnsprefix}.*" ]
then
restore_done="$(find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" -print -exec cp -pf "{}" "${adb_dnsdir}" \; 2>/dev/null)"
restore_done="$(find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" -print -exec cp -pf "{}" "${adb_dnsdir}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${restore_done}" ]
then
@ -560,13 +561,13 @@ f_restore()
#
if [ -n "${restore_done}" ] || [ -n "${rm_done}" ]
then
/etc/init.d/dnsmasq restart >/dev/null 2>&1
/etc/init.d/dnsmasq restart
sleep 1
dns_status="$(ps 2>/dev/null | grep "[d]nsmasq" 2>/dev/null)"
if [ -n "${dns_status}" ]
rc="$(ps | grep -q "[d]nsmasq"; printf ${?})"
if [ $((rc)) -eq 0 ]
then
rc=0
adb_count="$(head -qn -3 "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null | wc -l)"
adb_count="$(head -qn -3 "${adb_dnsdir}/${adb_dnsprefix}."* | wc -l)"
if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ]
then
adb_count="$((adb_count / 2))"
@ -599,11 +600,11 @@ f_exit()
#
if [ -f "${adb_tmpfile}" ]
then
rm -f "${adb_tmpfile}" >/dev/null 2>&1
rm -f "${adb_tmpfile}"
fi
if [ -d "${adb_tmpdir}" ]
then
rm -rf "${adb_tmpdir}" >/dev/null 2>&1
rm -rf "${adb_tmpdir}"
fi
# final log message and iptables statistics
@ -633,6 +634,6 @@ f_exit()
else
rc=0
fi
rm -f "${adb_pidfile}" >/dev/null 2>&1
rm -f "${adb_pidfile}"
exit ${rc}
}

+ 53
- 37
net/adblock/files/adblock-update.sh View File

@ -10,6 +10,15 @@
#
LC_ALL=C
# script debug switch (disabled by default)
# set 'DEBUG=1' to enable script debugging
#
DEBUG=0
if [ $((DEBUG)) -eq 0 ]
then
exec 2>/dev/null
fi
# pid handling
#
adb_pid="${$}"
@ -18,7 +27,7 @@ adb_pidfile="/var/run/adblock.pid"
if [ -r "${adb_pidfile}" ]
then
rc=255
/usr/bin/logger -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile} 2>/dev/null))"
/usr/bin/logger -s -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile}))"
exit ${rc}
else
printf "${adb_pid}" > "${adb_pidfile}"
@ -27,17 +36,18 @@ fi
# get current directory, script- and openwrt version
#
adb_scriptdir="${0%/*}"
adb_scriptver="0.91.0"
openwrt_version="$(cat /etc/openwrt_version 2>/dev/null)"
adb_scriptver="1.0.0"
openwrt_version="$(cat /etc/openwrt_version)"
# source in adblock function library
#
if [ -r "${adb_scriptdir}/adblock-helper.sh" ]
then
. "${adb_scriptdir}/adblock-helper.sh" 2>/dev/null
. "${adb_scriptdir}/adblock-helper.sh"
else
rc=254
/usr/bin/logger -t "adblock[${adb_pid}] error" "adblock function library not found"
/usr/bin/logger -s -t "adblock[${adb_pid}] error" "adblock function library not found"
rm -f "${adb_pidfile}"
exit ${rc}
fi
@ -71,13 +81,16 @@ then
shalla_file="${adb_tmpdir}/shallalist.txt"
src_name="shalla"
adb_dnsfile="${adb_dnsdir}/${adb_dnsprefix}.${src_name}"
list_time="$(awk '$0 ~ /^# last modified/ {printf substr($0,18)}' "${adb_dnsfile}" 2>/dev/null)"
if [ -r "${adb_dnsfile}" ]
then
list_time="$(awk '$0 ~ /^# last modified/ {printf substr($0,18)}' "${adb_dnsfile}")"
fi
f_log "=> (pre-)processing adblock source '${src_name}'"
# only process shallalist archive with updated timestamp,
# extract and merge only domains of selected shallalist categories
#
shalla_time="$(${adb_fetch} ${wget_parm} --server-response --spider "${adb_arc_shalla}" 2>&1 | awk '$0 ~ /Last-Modified/ {printf substr($0,18)}' 2>/dev/null)"
shalla_time="$(${adb_fetch} ${wget_parm} --server-response --spider "${adb_arc_shalla}" 2>&1 | awk '$0 ~ /Last-Modified/ {printf substr($0,18)}')"
if [ -z "${shalla_time}" ]
then
shalla_time="$(date)"
@ -85,14 +98,14 @@ then
fi
if [ -z "${list_time}" ] || [ "${list_time}" != "${shalla_time}" ]
then
${adb_fetch} ${wget_parm} --output-document="${shalla_archive}" "${adb_arc_shalla}" 2>/dev/null
${adb_fetch} ${wget_parm} --output-document="${shalla_archive}" "${adb_arc_shalla}"
rc=${?}
if [ $((rc)) -eq 0 ]
then
> "${shalla_file}"
for category in ${adb_cat_shalla}
do
tar -xOzf "${shalla_archive}" BL/${category}/domains 2>/dev/null >> "${shalla_file}"
tar -xOzf "${shalla_archive}" BL/${category}/domains >> "${shalla_file}"
rc=${?}
if [ $((rc)) -ne 0 ]
then
@ -103,8 +116,8 @@ then
# remove temporary files
#
rm -f "${shalla_archive}" >/dev/null 2>&1
rm -rf "${adb_tmpdir}/BL" >/dev/null 2>&1
rm -f "${shalla_archive}"
rm -rf "${adb_tmpdir}/BL"
if [ $((rc)) -eq 0 ]
then
adb_sources="${adb_sources} ${shalla_file}&ruleset=rset_shalla"
@ -138,7 +151,10 @@ do
url="${src/\&ruleset=*/}"
src_name="${src/*\&ruleset=rset_/}"
adb_dnsfile="${adb_dnsdir}/${adb_dnsprefix}.${src_name}"
list_time="$(awk '$0 ~ /^# last modified/ {printf substr($0,18)}' "${adb_dnsfile}" 2>/dev/null)"
if [ -r "${adb_dnsfile}" ]
then
list_time="$(awk '$0 ~ /^# last modified/ {printf substr($0,18)}' "${adb_dnsfile}")"
fi
f_log "=> processing adblock source '${src_name}'"
# prepare find statement with active adblock list sources
@ -154,12 +170,12 @@ do
#
if [ "${src_name}" = "blacklist" ]
then
url_time="$(date -r "${adb_blacklist}" 2>/dev/null)"
url_time="$(date -r "${adb_blacklist}")"
elif [ "${src_name}" = "shalla" ]
then
url_time="${shalla_time}"
else
url_time="$(${adb_fetch} ${wget_parm} --server-response --spider "${url}" 2>&1 | awk '$0 ~ /Last-Modified/ {printf substr($0,18)}' 2>/dev/null)"
url_time="$(${adb_fetch} ${wget_parm} --server-response --spider "${url}" 2>&1 | awk '$0 ~ /Last-Modified/ {printf substr($0,18)}')"
fi
if [ -z "${url_time}" ]
then
@ -170,14 +186,14 @@ do
then
if [ "${src_name}" = "blacklist" ]
then
tmp_domains="$(cat "${adb_blacklist}" 2>/dev/null)"
tmp_domains="$(cat "${adb_blacklist}")"
rc=${?}
elif [ "${src_name}" = "shalla" ]
then
tmp_domains="$(cat "${shalla_file}" 2>/dev/null)"
tmp_domains="$(cat "${shalla_file}")"
rc=${?}
else
tmp_domains="$(${adb_fetch} ${wget_parm} --output-document=- "${url}" 2>/dev/null)"
tmp_domains="$(${adb_fetch} ${wget_parm} --output-document=- "${url}")"
rc=${?}
fi
else
@ -194,7 +210,7 @@ do
f_log " source download finished (${count} entries)"
if [ "${src_name}" = "shalla" ]
then
rm -f "${shalla_file}" >/dev/null 2>&1
rm -f "${shalla_file}"
fi
unset tmp_domains
elif [ $((rc)) -eq 0 ] && [ -z "${tmp_domains}" ]
@ -220,10 +236,10 @@ do
then
if [ -s "${adb_whitelist}" ]
then
grep -Fvxf "${adb_whitelist}" "${adb_tmpfile}" 2>/dev/null | sort -u 2>/dev/null | eval "${adb_dnsformat}" 2>/dev/null > "${adb_dnsfile}"
grep -Fvxf "${adb_whitelist}" "${adb_tmpfile}" | sort -u | eval "${adb_dnsformat}" > "${adb_dnsfile}"
rc=${?}
else
sort -u "${adb_tmpfile}" 2>/dev/null | eval "${adb_dnsformat}" 2>/dev/null > "${adb_dnsfile}"
sort -u "${adb_tmpfile}" | eval "${adb_dnsformat}" > "${adb_dnsfile}"
rc=${?}
fi
@ -258,14 +274,14 @@ done
#
if [ -n "${adb_srclist}" ]
then
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srclist} \) -print -exec rm -f "{}" \; 2>/dev/null)"
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srclist} \) -print -exec rm -f "{}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${rm_done}" ]
then
f_log "disabled adblock lists removed"
if [ "${backup_ok}" = "true" ]
then
rm_done="$(find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srclist} \) -print -exec rm -f "{}" \; 2>/dev/null)"
rm_done="$(find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srclist} \) -print -exec rm -f "{}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${rm_done}" ]
then
@ -282,14 +298,14 @@ then
f_exit
fi
else
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" -print -exec rm -f "{}" \; 2>/dev/null)"
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" -print -exec rm -f "{}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${rm_done}" ]
then
f_log "all adblock lists removed"
if [ "${backup_ok}" = "true" ]
then
rm_done="$(find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" -print -exec rm -f "{}" \; 2>/dev/null)"
rm_done="$(find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" -print -exec rm -f "{}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${rm_done}" ]
then
@ -311,7 +327,7 @@ fi
#
if [ "${backup_ok}" = "true" ] && [ -n "${adb_errsrclist}" ]
then
restore_done="$(find "${adb_backupdir}" -maxdepth 1 -type f \( ${adb_errsrclist} \) -print -exec cp -pf "{}" "${adb_dnsdir}" \; 2>/dev/null)"
restore_done="$(find "${adb_backupdir}" -maxdepth 1 -type f \( ${adb_errsrclist} \) -print -exec cp -pf "{}" "${adb_dnsdir}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${restore_done}" ]
then
@ -333,35 +349,35 @@ then
# generate a temporary unique overall list
#
head -qn -3 "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null | sort -u 2>/dev/null > "${adb_dnsdir}/tmp.overall"
head -qn -3 "${adb_dnsdir}/${adb_dnsprefix}."* | sort -u > "${adb_dnsdir}/tmp.overall"
# loop through all separate lists, ordered by size (ascending)
#
for list in $(ls -Sr "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null)
for list in $(ls -Sr "${adb_dnsdir}/${adb_dnsprefix}."*)
do
# check original separate list vs. temporary overall list,
# rewrite only duplicate entries back to temporary separate list
#
list="${list/*./}"
sort "${adb_dnsdir}/tmp.overall" "${adb_dnsdir}/${adb_dnsprefix}.${list}" 2>/dev/null | uniq -d 2>/dev/null > "${adb_dnsdir}/tmp.${list}"
sort "${adb_dnsdir}/tmp.overall" "${adb_dnsdir}/${adb_dnsprefix}.${list}" | uniq -d > "${adb_dnsdir}/tmp.${list}"
# rewrite only unique entries back to temporary overall list
#
tmp_unique="$(sort "${adb_dnsdir}/tmp.overall" "${adb_dnsdir}/tmp.${list}" 2>/dev/null | uniq -u 2>/dev/null)"
tmp_unique="$(sort "${adb_dnsdir}/tmp.overall" "${adb_dnsdir}/tmp.${list}" | uniq -u)"
printf "%s\n" "${tmp_unique}" > "${adb_dnsdir}/tmp.overall"
# write unique result back to original separate list (with list footer)
#
tail -qn 3 "${adb_dnsdir}/$adb_dnsprefix.${list}" 2>/dev/null >> "${adb_dnsdir}/tmp.${list}"
mv -f "${adb_dnsdir}/tmp.${list}" "${adb_dnsdir}/${adb_dnsprefix}.${list}" >/dev/null 2>&1
tail -qn 3 "${adb_dnsdir}/$adb_dnsprefix.${list}" >> "${adb_dnsdir}/tmp.${list}"
mv -f "${adb_dnsdir}/tmp.${list}" "${adb_dnsdir}/${adb_dnsprefix}.${list}"
done
rm -f "${adb_dnsdir}/tmp.overall" >/dev/null 2>&1
rm -f "${adb_dnsdir}/tmp.overall"
fi
fi
# set separate list count & get overall count
#
for list in $(ls -Sr "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null)
for list in $(ls -Sr "${adb_dnsdir}/${adb_dnsprefix}."*)
do
list="${list/*./}"
count="$(head -qn -3 "${adb_dnsdir}/${adb_dnsprefix}.${list}" | wc -l)"
@ -382,10 +398,10 @@ done
#
if [ -n "${adb_revsrclist}" ] || [ -n "${rm_done}" ] || [ -n "${restore_done}" ]
then
/etc/init.d/dnsmasq restart >/dev/null 2>&1
/etc/init.d/dnsmasq restart
sleep 1
dns_status="$(ps 2>/dev/null | grep "[d]nsmasq" 2>/dev/null)"
if [ -n "${dns_status}" ]
rc="$(ps | grep -q "[d]nsmasq"; printf ${?})"
if [ $((rc)) -eq 0 ]
then
f_log "adblock lists with overall ${adb_count} domains loaded"
else
@ -401,7 +417,7 @@ fi
#
if [ "${backup_ok}" = "true" ] && [ -n "${adb_revsrclist}" ] && [ "$(printf "${adb_dnsdir}/${adb_dnsprefix}."*)" != "${adb_dnsdir}/${adb_dnsprefix}.*" ]
then
backup_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrclist} \) -print -exec cp -pf "{}" "${adb_backupdir}" \; 2>/dev/null)"
backup_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrclist} \) -print -exec cp -pf "{}" "${adb_backupdir}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${backup_done}" ]
then


+ 6
- 1
net/adblock/files/adblock.conf View File

@ -3,7 +3,7 @@
config adblock 'global'
option adb_enabled '1'
option adb_cfgver '0.91'
option adb_cfgver '1.0'
option adb_blacklist '/etc/adblock/adblock.blacklist'
option adb_whitelist '/etc/adblock/adblock.whitelist'
option adb_forcedns '1'
@ -72,6 +72,11 @@ config source 'spam404'
option adb_src 'https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt&ruleset=rset_spam404'
option adb_srcdesc 'generic blocklist for suspicious domains, infrequent updates, approx. 5.000 entries'
config source 'sysctl'
option enabled '0'
option adb_src 'http://sysctl.org/cameleon/hosts&ruleset=rset_sysctl'
option adb_srcdesc 'generic blocklist for ad related domains, weekly updates, approx. 21.000 entries'
config source 'whocares'
option enabled '0'
option adb_src 'http://someonewhocares.org/hosts/hosts&ruleset=rset_whocares'


+ 8
- 7
net/adblock/files/adblock.init View File

@ -3,6 +3,7 @@
START=99
exec 2>/dev/null
adb_pid="${$}"
adb_script="/usr/bin/adblock-update.sh"
adb_pidfile="/var/run/adblock.pid"
@ -19,7 +20,7 @@ fi
if [ -r "${adb_pidfile}" ]
then
"${adb_logger}" ${log_parm} -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile} 2>/dev/null))"
"${adb_logger}" ${log_parm} -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile})" 2>&1
exit 255
fi
@ -48,23 +49,23 @@ reload()
stop()
{
rm_done="$(find "/tmp/dnsmasq.d" -maxdepth 1 -type f -name "adb_list.*" -print -exec rm -f "{}" \; 2>/dev/null)"
rm_done="$(find "/tmp/dnsmasq.d" -maxdepth 1 -type f -name "adb_list.*" -print -exec rm -f "{}" \;)"
rc=${?}
if [ $((rc)) -eq 0 ] && [ -n "${rm_done}" ]
then
/etc/init.d/dnsmasq restart >/dev/null 2>&1
/etc/init.d/firewall restart >/dev/null 2>&1
/etc/init.d/dnsmasq restart
/etc/init.d/firewall restart
fi
uhttpd_pid="$(ps 2>/dev/null | grep "[u]httpd.*\-h /www/adblock" 2>/dev/null | awk '{print $1}' 2>/dev/null)"
uhttpd_pid="$(ps | grep "[u]httpd.*\-h /www/adblock" | awk '{print $1}')"
if [ -n "${uhttpd_pid}" ]
then
kill -9 "${uhttpd_pid}" >/dev/null 2>&1
kill -9 "${uhttpd_pid}"
fi
if [ -n "${rm_done}" ] || [ -n "${uhttpd_pid}" ]
then
"${adb_logger}" ${log_parm} -t "adblock[${adb_pid}] info " "all adblock related services stopped"
"${adb_logger}" ${log_parm} -t "adblock[${adb_pid}] info " "all adblock related services stopped" 2>&1
fi
return 0
}

Loading…
Cancel
Save