[Rspamd-Users] New Spamhaus zone and updates to the rules

[Riccardo Alfieri]

On 30/04/20 11:26, Vsevolod Stakhov wrote:

> Unfortunately you have not considered my suggestions and I had no time
> to fix and test these zones by myself.

Hello,

be assured that the team discussed what you shared with my colleague, 
I'll explain what it did come out of that discussion

> 1) That should be a part of RBL module! With crazy usage of
> `r:resolve_a({ task = task, name = lookup , callback = dns_cb, forced =
> true })` it is trivial to create a message that would kill your DNS
> infrastructure after a few emails.
> So the only way to do it properly is to reuse bells and whistles from
> RBL module (probably that would require patching) and selectors framework.
> All other ways are not good from architectural point of view and suffer
> from lot's of mistakes.
At the moment we are studying the selectors framework, I hope we'll be 
able to update the code in the near future
> 2) `local re =
> rspamd_re.create_cached('^(?:bc1|[13])[a-zA-HJ-NP-Z0-9]{25,39}$')`
> should not be used to match regular expressions: this is not optimized
> by hyperscan and you can easily find yourself running in an semi-endless
> backtracking loop with PCRE. So it is a big no way for production usage.
> Furthermore, Rspamd has `BITCOIN_ADDR` symbol that can detect and
> validate some of the popular bitcoin formats (it uses the same approach
> as your rule but this needs to be changed tbh). Notably exceptions are
> ETH and monero wallets - these are NYI. However, BTC and BTC cache
> (segwit) addresses are fully supported.
> So there is no need to call this expensive RE multiple times I suppose -
> it is better to refactor and improve BITCOIN_ADDR symbol and use it in
> the selectors based RBL.
Here we will have to basically use two different approaches, meaning 
that for addresses included in BITCOIN_ADDR we would be able to use what 
Rspamd already gives us, but for the others addresses we would still 
need to support the slower extraction method. I think that could be 
done, I'll take a look on how to use BITCOIN_ADDR symbol
> 3) Rspamd now supports RFC version of base32 out of the box, so no need
> to use Lua for string transformation - it is not a good idea in general.

Thats great, but I'm unable to find the function name :( can you maybe 
point me to where to look? And from what version of Rspamd is that 
supported?

Also, we may need to keep the slower LUA version for older Rspamd 
releases.. unfortunately not everyone is on the latest version

> 4) Sha256 is quite a bad choice if it comes to Rspamd. Ideally would be
> to have a blake2b zone of hashes together with sha256 for other
> appliances. The reason to do it aside of the overall speed of blake2b
> comparing to sha256 is that blake2b hash is already calculated for all
> content parts. Not sure if that's doable.
As you probably noticed, we moved CW and emails lookups to SHA1 , 
leaving only the file component to BASE32(SHA256) lookups. I don't think 
blake2b would be supported by us in the future but if it does be sure 
we'll update the lookups
> I'm not quite sure that I would have time to fix these issues in short
> terms I'm afraid. However, the current rules set seems to be not ready
> for the production usage.

We have an Rspamd installation with HBL on a single server with 2cpu and 
4GB RAM that does ~100k emails/day and it works just fine. I know 100k 
emails/day are a very low volume, but given the cpu/ram usage history I 
think that server could push to 500k without the need to an hardware 
upgrade.

Obviously these are our observations and should be taken with a grain of 
salt :)

Anyway, thanks for the feedback and suggestions, again be reassured that 
we are taking them seriously

-- 
Best regards,
Riccardo Alfieri

Spamhaus Technology
https://www.spamhaustech.com/