[Rspamd-Users] Rspamd 2.1 has been released

Thu Oct 31 10:32:31 UTC 2019

Thank you very much for your work!

However, something is still not perfect. Today, shortly after the
neural module finished learning, I observed some similar messages in
rspamd.log:

2019-10-31 08:54:22 #25914(normal) rspamd_crash_sig_handler: caught
fatal signal 11(Segmentation fault), pid: 25914, trace:
2019-10-31 08:54:22 #25912(rspamd_proxy) <485876>; proxy;
proxy_backend_master_error_handler: abnormally closing connection from
backend: 127.0.0.1, error: IO read error: unexpected EOF, retries
left: 5
2019-10-31 08:54:22 #25912(rspamd_proxy) <485876>; proxy;
proxy_backend_master_error_handler: retry connection to: 127.0.0.1
retries left: 4
2019-10-31 08:54:22 #25898(main) <62e687>; main;
rspamd_check_termination_clause: normal process 25914 terminated
abnormally by signal: Segmentation fault and created core file

While this appeared in dmesg:

[237212.003348] traps: rspamd[31649] trap invalid opcode
ip:7fb957031f88 sp:7ffdee9365b8 error:0 in #129734
(deleted)[7fb957026000+10000]
[237212.056962] traps: rspamd[31650] trap invalid opcode
ip:7fb957031f88 sp:7ffdee9365b8 error:0 in #129734
(deleted)[7fb957026000+10000]
[237214.067469] traps: rspamd[31653] trap invalid opcode
ip:7fb957031f88 sp:7ffdee9365b8 error:0 in #129734
(deleted)[7fb957026000+10000]
[237214.114069] traps: rspamd[31654] trap invalid opcode
ip:7fb957031f88 sp:7ffdee9365b8 error:0 in #129734
(deleted)[7fb957026000+10000]
[237216.147868] traps: rspamd[31657] trap invalid opcode
ip:7fb957031f88 sp:7ffdee9365b8 error:0 in #129734
(deleted)[7fb957026000+10000]
[237216.190807] traps: rspamd[31658] trap invalid opcode
ip:7fb957031f88 sp:7ffdee9365b8 error:0 in #129734
(deleted)[7fb957026000+10000]

And after this rspamd ceased working. A manual restart brought it back
to life however, and after this restart even the neural module started
to classify emails. So, it seems to me that as soon as the neural
module "knows enough", it crashes, and rspamd has to be restarted in
order to work again.

Best wishes,

NÉMETH, Tamás

Vsevolod Stakhov <vsevolod at rspamd.com> ezt írta (időpont: 2019. okt.
28., H, 22:32):
>
> We have released Rspamd 2.1 today.
>
> This release contains some new features and many bug fixes. There are no
> incompatible changes introduced with this release to our best knowledge.
>
> This release includes the following features and important fixes.
>
>
> Add uuencode support
>
> Despite of being very old standard, UUenconde parts are still quite
> common in the email traffic observed. From this version, Rspamd supports
> uuencoded parts (both normal and base64 version).
>
>
> Critical issue found in dkim verification
>
> There was a critical regression in 2.0 DKIM verification code caused
> verification failures for some of the valid DKIM signatures. More
> details are in the GitHub issue.
>
>
> Improved neural training
>
> There are number of fixes and improvements in the Neural module. Now all
> training samples are balanced using random sampling allowing a smoother
> training vectors selection. Some number of bugs has been fixed, as well
> as scores are NO LONGER RECOMMENDED to select training vectors - Rspamd
> automatically applies heuristic to select messages for learning. Also
> some issues around infinities and learning threads count have been
> addressed.
>
>
> Maps fixes
>
> There are number of fixes and improvements around maps handling logic.
> This include fixes for both HTTP and file maps, as well as better
> timeout and caches handling.
>
>
> Event loop fixes
>
> Rspamd could previously select an inefficient backend on some OSes,
> notably, on BSD and OSx. This version should fix it. The ability to
> configure the events backend manually via the configuration file has
> also been added to Rspamd.
>
>
> Full list of the meaningful changes
>
> -   [Conf] Update neural.conf
> -   [CritFix] Fix dkim verification for multiple headers listed
> -   [Feature] Add support of uudecode
> -   [Feature] Allow to explicitly set events backend
> -   [Feature] Implement configurable limits for SPF lookups
> -   [Feature] Lua_scanners: Use lua magic for inclusion/exclusion logic
> -   [Feature] Multimap: Do not check files in office archives
> -   [Feature] Neural: Add sampling when storing training vectors
> -   [Feature] SPF: Allow to disable AAAA checks in configuration
> -   [Feature] Spf: Add limits configuration support
> -   [Feature] Store etag in cached HTTP maps + better logging
> -   [Feature] Support segwit BTC addresses, fix LTC verification
> -   [Feature] Support uuencoding
> -   [Fix] Add configurable number of threads for OpenBLAS
> -   [Fix] Add workaround for ragel 7 in hyperscan related maps code
> -   [Fix] Another fix for numeric urls parsing
> -   [Fix] Correct EMA time calculations
> -   [Fix] Do not treat archives as text
> -   [Fix] Do not use strdup on data extracted from lua
> -   [Fix] Fix a failure calcuating URL reputation.
> -   [Fix] Fix crash due to constructors init order
> -   [Fix] Fix crash on parts with no cd
> -   [Fix] Fix empty prefilters that require mime structures
> -   [Fix] Fix event loop creation
> -   [Fix] Fix issues sending DMARC reports.
> -   [Fix] Fix misprint
> -   [Fix] Fix saving of the file maps
> -   [Fix] Fix size calculations when converting from utf16
> -   [Fix] Fix support of disable_monitoring in rbl
> -   [Fix] Fix use-after-free
> -   [Fix] Fix zip files check to relax requirements
> -   [Fix] Important hiredis fixes
> -   [Fix] Lot’s of fixes in maps check logic
> -   [Fix] Lua_tcp: Deal with temporary fails on write
> -   [Fix] Lua_tcp: Make write errors fatal and rework error handlers
> -   [Fix] Meta: Filter some more values
> -   [Fix] Neural: Add protection agains infinities
> -   [Fix] Oops, fix math.huge invocation
> -   [Fix] Plug memory leak
> -   [Fix] Sigh, another email to string fix
> -   [Fix] Try to fix another ownership race in ssl connection
> -   [Fix] Uuencode: Fix parsing of corrupted uuencode
> -   [Fix] lua_scanners - razor rename need_check function
> -   [Rework] Require CMake 3.9 to work, remove manual lto crap
> --
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users