[Rspamd-Users] Rspamd 1.9.1 has been released

[Sophie Loewenthal]

   Hi,
   How does LEAKED_PASSWORD_SPAM   distinguish etterbeek legitimate
   bitcoin addresses and ones used for scamming?

   On Fri, Apr 5, 2019 at 12:37 PM Vsevolod Stakhov <vsevolod at rspamd.com>
   wrote:

     We have released Rspamd 1.9.1 today.

     This release includes one potentially dangerous change: all
     configuration files are now preprocessed using Jinja templates.

     Hence, if you have sequences like {=/=}, or {%/%}, or {#/#}
     ANYWHERE in the configuration files including even comments then you
     need to take extra care when moving these configuration to the new
     version! There are workarounds described above to do that. {% endraw
     %}

     Here is the list of the most important changes in this version.

     Jinja templates in the configuration

     From version 1.9.1, Rspamd supports Jinja2 templates provided by
     Lupa
     Lua library. You can read the basic syntax documnentation and the
     abilities provided by these templating engines using the links
     above.
     Rspamd itself uses a specific syntax for variable tags: {= and =}
     instead of the traditional {{ and }} as these tags could mean, e.g.
     a
     table in table in Lua.

     Templating might be useful to hide some secrets from config files
     and
     places them in environment. Rspamd automatically reads environment
     variables that start from RSPAMD_ prefix and pushes it to the env
     variable, e.g. RSPAMD_foo=bar comes to env.foo="bar" in templates.

     New template subcommand in Rspamadm

     Rspamadm has now template subcommand to apply templates engine to
     the
     input file or files:

     Options supported:

     |? |-n, ?no-vars | Don?t add Rspamd internal variables | | -e , ?env
     |
     Load additional environment var from specific file (name=value) | |
     -l ,
     ?lua-env | Load additional environment vars from specific file (lua
     source) | | -s , ?suffix | Store files with the new suffix | | -i,
     ?inplace | Replace input file(s) |

     Changes in URLs extraction for HTML parts

     Rspamd now tries to extract URLs from plain text of HTML parts.
     Unfortunately, despite of being contraversal, some Email clients do
     that
     as well. One of the notable example is Outlook. Hence, from this
     release
     Rspamd also looks for URLs in plain HTML text.

     Per user settings for mime_types plugin

     Mime types plugin now supports per user settings to allow individual
     black and white lists of extensions. Here is an example to increase
     score for exe extensions for some specific user:

     test {
     from = "user at example.com";

     apply {
     plugins {
     mime_types = {
     bad_extensions = {
     exe = 100500,
     }
     }
     }
     }
     }

     Mime types plugin now also supports reverse mapping of content type
     to
     extension to allow processing of attachments where an exact file
     name is
     not specified.

     Better greylisting conditioning

     It is now possible to disable or enable greylisting in Rspamd based
     on
     the presence of some specific symbols. This feature allows more fine
     grained greylisting control.

     Bitcoin addresses validation

     It is not a secret that the wave of spam and scam related to crypto
     currencies has been flooding the email flows in the recent time.
     Rspamd
     has a special rule called LEAKED_PASSWORD_SPAM to block some of the
     scam
     types. In this version, Rspamd also checks bitcoin wallets to
     distinguish them from random long strings to reduce false positives
     rate
     significantly. It also allows to build a database of wallets used
     for
     scam and spam.

     Replies plugin validation

     Replies plugin now stores the from/reply-to address when tracking
     outbound messages and whitelists merely replies that come that
     address.
     It helps to avoid replies abusing where spammers were able to catch
     some
     legit message ids somewhere in public lists and used them in
     In-Reply-To
     headers to dodge spam filtering in Rspamd.

     List of major bug fixes

     This version includes some important fixes:

     - Add crash safety for HTTP async routines
     - Clickhouse: Fix table schema upload
     - Core: Fix squeezed dependencies handling for virtual symbols
     - Finally fix default parameters parsing in actions section
     - Fix ES sending logic (restore from coroutines mess)
     - Fix finishing script for Clickhouse collection
     - Fix priority for regexp symbols registration
     - Neural: Fix training
     - Rework cached Redis logic to avoid sentinels breaking
     - SURBL: Fix regression in surbl module
     - Fix double signing in the milter

     Full list of the meaningful changes

     - [Conf] Add vendor groups for symbols
     - [Feature] Add rspamadm template command
     - [Feature] Allow to add messages from settings
     - [Feature] Allow unconnected DNS servers operations
     - [Feature] Check limits after being set, migrate to uint64
     - [Feature] Greylist: Allow to disable greylisting depending on
     symbols
     - [Feature] Improve lua binary strings output
     - [Feature] Mime_types: Implement user configurable extension
     filters
     - [Feature] Mime_types: When no extension defined, detect it by
     content
     - [Feature] Preprocess config files using jinja templates
     - [Feature] Replies: Filter replies sender to limit whitelisting to
     direct messages
     - [Feature] Treat all tags with HREF as a potential hyperlinks
     - [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM
     - [Fix] Add crash safety for HTTP async routines
     - [Fix] Another fix for Redis sentinel
     - [Fix] Clickhouse: Fix table schema upload
     - [Fix] Core: Fix squeezed dependencies handling for virtual symbols
     - [Fix] Finally fix default parameters parsing in actions section
     - [Fix] Fix ES sending logic (restore from coroutines mess)
     - [Fix] Fix finishing script for clickhouse collection
     - [Fix] Fix priority for regexp symbols registriation
     - [Fix] Fix various issues found by PVS Studio
     - [Fix] Initialize lua debugging earlier
     - [Fix] Neural: Fix training
     - [Fix] Rework cached Redis logic to avoid sentinels breaking
     - [Fix] SURBL: Fix regression in surbl module
     - [Fix] Fix double signing in the milter
     - [Project] Add support of HTTP proxy in requests
     - [Rework] Change lua global variables registration
     - [Rework] Rework HTML content urls extraction
     - [Rework] Start rework of aliasing in Rspamd
     - [WebUI] Combine Scan and Learning into one tab
     - [WebUI] Fix symbol score input type
     - [WebUI] Show grayed out pie
     - [WebUI] Update Throughput summary values dynamically
     --
     Users mailing list
     Users at lists.rspamd.com
     https://lists.rspamd.com/mailman/listinfo/users

   --

   here we come with the sound of a drum