[Rspamd-Users] Rspamd 2.0 has been released
azurit at pobox.sk
azurit at pobox.sk
Wed Oct 16 06:20:57 UTC 2019
thank you very much for your work!
In the message below, you mentioned that users or ratelimit module
should read the upgrading instructions (is this it?
https://rspamd.com/doc/migration.html#migration-to-rspamd-20 ). I read
it and didn't find anything related to ratelimit module. Is something
Citát Vsevolod Stakhov <vsevolod at rspamd.com>:
> We have released Rspamd 2.0 today!
> This version encompasses new versioning schema that will be used in
> future Rspamd releases: specifically, instead of the
> <major>.<minor>.<patch>, Rspamd will use just <major>.<minor> versioning
> schema. This happens because the <major> number has never been increased
> for many years and <minor> number has been used as a real version indicator.
> Upgrade notes
> There are various important features in this release. The vast majority
> of those should not have any visible impact on the existing systems.
> However, you are recommended to read the Upgrade Notes.
> The main potential source of incompatibilities is the deprecation of the
> surbl and emails modules that have been replaced with rbl module. The
> default Bayes backend is also changed to Redis now while the Sqlite
> backend is now marked as deprecated and is not recommended for use.
> ip_score, neural and ratelimit modules users are strongly advised to
> read the upgrading notes!
> Packages support
> In this version of Rspamd, we have stopped support of the following OS
> - Ubuntu trusty (reached EOL)
> - Centos 6 (almost reached EOL)
> We have added Centos 8 packages instead.
> As usually, Rspamd project strongly recommends NOT TO USE the packages
> that are provided by 3rd parties, including your own LINUX DISTRIBUTION.
> These packages are usually out-of-date, built incorrectly and
> accordingly, they are not supported by Rspamd project. Please use the
> official packages only. FreeBSD ports are considered official packages
> as they are supported by Rspamd project directly (well, strictly
> speaking by myself).
> Here is the list of the most important changes in this release.
> Libevent has been replaced with bundled libev
> After many years of using the libevent library Rspamd switched to libev
> library. The main reason was performance and control: there were many
> libevent versions shipped with various supported platforms and many of
> those lacked important features, such as inotify support for Linux.
> Switching to libev allowed us to simplify the code, improve signals
> handling, improve timeouts handling and deal with file maps changes
> instantly due to inotify.
> Torch has been dropped from Rspamd
> Lua torch has served as a powerful engine for ML and neural networks in
> Rspamd for quite a long time. However, it is no longer maintained or
> updated and its support has proven to be a nightmare. There were also
> important bugs that could not be fixed due to the code complexity. From
> version 2.0, Rspamd adopted kann library that is much more friendly for
> embedding and provides very convenient interfaces that are now exported
> via Lua.
> RBL module improvements and replacement of the SURBL and Emails module
> RBL module has replaced both emails and surbl modules unifying all
> Runtime Black Lists checks in a single place. It has added new RBL
> types, such as selectors, and the simplified extending of the existing
> rules to more powerful ones.
> Emails rules with maps instead of DNS RBLs are NO LONGER SUPPORTED.
> Please use multimap with selectors instead.
> New Lua Magic library
> For file types detection, Rspamd now uses an own implementation of
> detection library based on Lua and Hyperscan (where possible) instead of
> libmagic. There are 4 major concerns for that:
> - Libmagic is a generic library that can easily detect pdp11 a.out
> format but can fail in docx detection surprisingly often
> - We need performance and libmagic is not about performance at all
> - We want to add new detection heuristics instead of relying on 3rd
> party strict rules
> - Libmagic API is not very suitable for us
> With the new library, Rspamd can detect part types in just a couple of
> microseconds and find the vast majority of the _interesting_ things,
> such as executables, archives, images, html and so on and so forth.
> Neural module rework
> Neural networks module has been almost totally rewritten to support KANN
> library and symbols profiles. Now, Rspamd will not reset neural network
> on each individual symbol change - it will try to use the most
> appropriate network instead. Many issues with neural learning dead locks
> have also been addressed.
> Clickhouse module improvement
> - Added LowCardinality fields to improve storage requirements
> - Fixed retention code
> - Significantly optimized memory usage by using userdata instead of
> interned strings
> Multimap module
> Various new features, including maps combinations and
> dependent maps(https://rspamd.com/doc/modules/multimap.html#dependent-maps).
> Maillist module
> Improved mailing lists detection and reworked detection heuristic.
> Heartbeats support
> Rspamd workers now send heartbeat events to the main process. In turn,
> the main process can now kill hanged workers if a reasonable amount of
> heartbeats have been lost. This feature is not enabled by default for
> Lua scanners improvements
> There are lots of additions in lua scanners. Many of those have been
> contributed by Carsten Rosenberg from HeinleinSupport.
> New antivirus engines support:
> - Kaspersky ScanEngine: https://www.kaspersky.com/scan-engine
> - Trend Micro IWSVA support via icap (by @c-rosenberg)
> - F-Secure Internet Gatekeeper via icap (by @c-rosenberg)
> New external scanners:
> - Razor support (by @c-rosenberg)
> - Better oletools support (by @c-rosenberg)
> - P0F support as a separate module (by Denis Paavilainen -
> Mime modifications
> From version 2.0, Rspamd allows modifying messages via Lua API methods.
> This support required massive rework of the internal structures and have
> been tested by Migadu. These functions are implemented in the lua_mime
> Users settings improvements
> Rspamd now treats settings differently if they are set via Settings-Id:
> there are certain performance benefits and better logging in all
> modules. It is also possible to bind rules explicitly to certain
> settings id allowing to separate mail processing flows more efficiently.
> Upstreams library improvements
> - Added lazy resolving of the upstreams
> - Added SRV upstreams to resolve SRV records for both names, ports,
> and priorities (e.g. by using Hashicorp Konsul DNS)
> - Use random strings for monitoring sanity
> Performance improvements
> - Improved base64 decoding for typical outputs
> - Langdet: Limit number of stop words to be checked
> - Added sanity limit for task:get_urls() method to avoid Lua memory
> - Maps: Allow caching for complex maps
> - Settings fast path have been added
> - Lua core: use lightuserdata to index classes to avoid strings
> - HTTP(s) keep-alive support has been added
> Rules and other improvements
> - Added BITCOIN_ADDR symbol to allow custom composite rules creation
> to block scam campaigns
> - Support Litecoin addresses
> - Implement syntax highlighting for Lua
> - Allow execution of async events when hs compiles regexps
> - Bayes expiry: eliminate default expiration mode (use lazy mode all
> the time)
> - Eliminate lua_squeeze as it has shown no improvements
> - Drop url tags
> - Eliminate virtual scan time as it is useless
> - Use replxx instead of linenoise
> - Added SSL/STARTTLS support to lua_tcp library
> - Implemented SSL graceful closing
> This version of Rspamd contains a number of other minor and major
> improvements and fixes compared to the 1.9 branch. This includes some
> bugs that were fixed in 2.0 and that could cause certain issues, hangs
> or crashes with certain emails.
> Users mailing list
> Users at lists.rspamd.com
More information about the Users