################################################################### # unbound-adblock 0.6 Release Notes / Changelog # Copyright 2018-2021 Jordan Geoghegan ################################################################### The unbound-adblock v0.6 release features a number of usability, efficiency and performance improvements. Release Notes: # Significant efficiency and performance improvements accross the board: - Improvements to URL Fetching: * Add support for aria2c HTTP/FTP fetch utility. * Add support for HTTP compression when using aria2c, curl or wget for fetching block lists. * We now opportunistically make use of the best available URL fetch utility, with overrides enabled via long options. * aria2c, curl and wget are now fed URLs over a pipe, which eliminates issues with argument length limits and maximizes potential socket and cache reuse. * Curl now properly follows redirects. * Enforce strict timeout limits to avoid hangs or infinite reads when fetching blocklists (functionality varies amoung supported URL fetch utilities). * FreeBSD/DragonflyBSD "fetch" utility now supports custom user agents. * Explicitely disable URL globbing. - Use parallelism/concurrency where appropriate: * Blocklist URLs can now be fetched in parallel thanks to aria2c. * Blocklist parsing is now performed in parallel. - New Features: * Add support for custom DNS blocking response codes: NXDOMAIN, NODATA, REFUSED and NULL. * Add '-C' flag to specify a configuration file to load. * Add '-P' flag to control maximum number of parallel processes. * Add '-U' option to specify HTTP user agent. * Add '-f' flag to load blocklist from local file system. * Add '-o' long options to set awk/grep/fetch utility preference. Please see manpage for details. * Add '-o print-debug' to print debugging information and statistics. - To make things faster, do less work: * Address parser is now less picky about list formatting. This allows for a broader range of blocklist formats to be ingested. * Mixed lists are now supported: Blocklists may now be comprised of any combination of supported blocklist formats, with no format declarations required. * Changes to the blocklist parsing/handling code has allowed for the removal of a bunch of superflous code. * As a result of the address parser refactoring, significant performance gains should be expected. * Multiple internal functions have been adjusted to avoid fork+exec and subshell overhead. * We now make use of modern shell features to eliminate numerous potential footguns. - Portability Improvements: * Communication with Unbound now occurs exclusively via unbound-control. This makes portability much easier, and allows for the removal of a bunch of code. * By default we now write generated blocklist data to symlink '/var/unbound-adblock/block.db'. This makes portability much easier, and allows for the removal of a bunch of code. - Configuration Changes: * Use of 'set -o pipefail' is now required. This means you'll need a modern oksh, bash, zsh or ksh93 shell to run unbound-adblock. * OS type decleration with the '-O' flag has been removed, as OS type is now auto-detected. * The '-o [no-]whitelist flags were renamed to '-o [no-]custom-filter' to reflect the features versatility. * Compressed lists may now only be declared in the "Custom Lists" section from a config file specified with '-C'. * Use of doas/sudo is now determined automatically, with doas being preferred. Use long options to override. --- * Remove all reference to /etc/hosts and domain-only list formats as they are now processed identically and no longer require differentiation from the users perspective. As a result, the '-t' and '-d' options have been removed entirely, and the '-l' and '-u' options repurposed. The '-l' option is now used to reference a path to a list of URLs and the '-u' option is used to reference a single URL. Please see the man page for details. --- * User Configuration Area Changes: The "User Configuration Area" at the top the script has been relocated to an external file. Config file path may be specified with the '-C' flag. This change allows for users to easily switch between a command line or config file driven workflow. As such, unbound-adblock may now be used as a versatile commandline utility with many potential use cases. If no config file is specified, then all configuration will be performed from the commandline. - Correctness: * Use specific integer/boolean checks to avoid undefined behavior. * Implemented better regular expressions to validate domain names. * We now properly handle Windows line endings (carriage returns). * Make sure ZSH doesn't clear the 'set -efo pipefail' options. - Misc: * Add support for whitelisting URL's with the unbound local-data backend. * Add support for 'ugrep' - a fast, portable, multithreaded grep replacement written in modern C++. * Add support for "goawk", a POSIX compliant awk implementation written in Go.