[Rspamd-Users] Rspamd 1.8.1 has been released

[Vsevolod Stakhov]

We have released RSPAMD 1.8.1 today.

There are couple of the incompatibilities introduced, hence, please
check the migration notes. These incompatibilities could affect almost
any Rspamd installation with custom configuration, so please read the
upgrade notes carefully.


The most important features and fixes

-   Critical fixes in DMARC module

    DMARC module could insert temporary failure symbol instead of strong
    rejection in certain cases. This has been fixed in 1.8.1. There are
    many other bugs fixed in this module since the whole check logic has
    been rewritten in accordance to RFC.

-   Fuzzy check

    Due to bug introduced in 1.8.0, there algorithm used to
    deterministically skip words in large text parts was not
    deterministic. It means that the exact words pipelines produced by
    different Rspamd instances might be different. It could affect if
    your words_limit was reached (default: words_decay = 200 words).
    Hence, for large text parts it was expected to have misses in fuzzy
    and in Bayes classification. Whilst bayes missing should not be
    significant, the fuzzy misses might be very severe and they might
    break fuzzy detection for large text parts.

    In 1.8.1, we have fixed this issue and, since we have already broken
    the compatibility with 1.7.9, we have decided to increase
    words_decay to 600. Please ensure that you don’t override this
    parameter anywhere (e.g. in local.d/options.inc,
    override.d/options.inc or any other override or local file) or your
    compatibility with Rspamd fuzzy storage would be lost for messages
    with more than words_decay threshold words.

-   Various language detection issues have been fixed
-   Experimental clustering plugin
-   Important fixes for the dynamic ratelimits
-   Fix options insertion
-   Plug memory leak in redis pool
-   Add check_violation feature to DKIM/ARC signing to avoid signing
    messages when there is an existing invalid DKIM signature
-   Add only unique elements to Clickhouse url arrays
-   Allow g+: and g-: composite atoms to include symbols with positive
    score and negative score
-   Allow dkim domains check in surbl module
-   Allow maps with HTTP auth using standard URL syntax
-   Allow to disable actions by users settings by setting them to null
-   Extend whitelisting options:
    -   Allow per element whitelist/blacklist only behaviour
    -   Introduce three types of lists:
        -   Blacklists (bl)
        -   Whitelists (wl)
        -   Bidirectional lists (both)
-   Use heuristical verdict instead of the plain action in plugins to
    detect if a message is clearly spam, junk, ham or uncertain when
    there is not enough confidence despite of the action being set
-   Various Web Interface improvements and fixes driven by Alexander
    Moisseev


Full list of the meaningful changes

-   [CritFix] Fix options insertion
-   [CritFix] Fix words decay one more time (affects long messages)
-   [CritFix] Increase default words_decay
-   [CritFix] Plug memory leak in redis pool
-   [Feature] Add check_violation feature to DKIM/ARC signing
-   [Feature] Add only unique elements to Clickhouse url arrays
-   [Feature] Allow g+: and g-: composite atoms
-   [Feature] Allow dkim domains check in surbl
-   [Feature] Allow maps with HTTP auth
-   [Feature] Allow to disable actions by users settings
-   [Feature] Extend whitelisting options
-   [Feature] Store url object in images
-   [Feature] Use verdict instead of the plain action in plugins
-   [Fix] Allow to call fstring append with NULL string
-   [Fix] DCC - luacheck
-   [Fix] Do not load torch on each rspamadm invocation
-   [Fix] Fix boundaries detection and rework stop words algorithm
-   [Fix] Fix dependencies for DNS_SIGNED symbol
-   [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
-   [Fix] Fix groups mess
-   [Fix] Fix groups mess
-   [Fix] Fix parsing address with comments
-   [Fix] Fix resolving in DMARC reports
-   [Fix] Fix various issues with parsing of the received headers
-   [Fix] Fix watchers issue in lua_tcp when doing no resolving
-   [Fix] Plug memory leak in language detector (affects reloads)
-   [Fix] Remove one letter stop words
-   [Fix] Slashing: backport chunk logic from libucl
-   [Fix] Stop libevent from using cached time in rspamadm
-   [Fix] Try to fix watchers chaining
-   [Fix] Various fixes in redis sync interface
-   [Fix] ip_score - respect check_authed and check_local settings from
config
-   [Project] Rework passthrough actions
-   [Project] Clustering module
-   [Rework] Always create result for a task
-   [Rework] Completely rewrite DMARC checks logic
-   [Rework] Rework and fix whitelist plugin
-   [WebUI] Add symbols sorting buttons
-   [WebUI] Change symbols order without updating history
-   [WebUI] Colorize symbols
-   [WebUI] Do not display password form when secure_ip is set
-   [WebUI] Fix symbol description tooltips display
-   [WebUI] History: add sorting by symbol score value