From privat at arnowelzel.de Wed Jul 1 11:55:26 2020 From: privat at arnowelzel.de (Arno Welzel) Date: Wed, 1 Jul 2020 13:55:26 +0200 Subject: [Rspamd-Users] Multiple dkim selectors without using a map? In-Reply-To: <20200630152353.l72x2vfhqul5gowa@shamm.as> References: <20200630152353.l72x2vfhqul5gowa@shamm.as> Message-ID: Hi, George Shammas via Users: > I have about 15 domains that can send email from my server. They are > currently all using rsa keys, but I want to add Ed25519 signatures to > all of them. > > Right now, the rspamd config is simple, and allows me to have per > domain keys pretty easily: ``` path = > "/var/lib/rspamd/dkim/$domain.$selector.key"; selector = "shammas"; > ``` > > However, it seems that if I want to sign with two keys, I'll need to > create a long map with all the domains, their selectors, and their > paths. At least that is my interpretation from the docs at > https://rspamd.com/doc/modules/dkim_signing.html [...] There are several ways to do this, not only maps. Another way would be to create a script which produces this map file based on the list of domain names. Arno From chris at ckruijntjens.nl Wed Jul 1 16:58:03 2020 From: chris at ckruijntjens.nl (Chris Kruijntjens) Date: Wed, 1 Jul 2020 16:58:03 +0000 Subject: [Rspamd-Users] rspamd action.conf "Reject null;" now working Message-ID: Hi All, I noticed if in the ovveride.d/actions.conf i enter the following mail is still being rejected. I dont want mail to be rejected. reject = null; add_header = 6; rewrite_subject = 7; greylist = null; ? ? mail still rejected. ? ? What am i doing wrong? Vriendelijke groet, Chris Kruijntjens. From rspamd at shamm.as Thu Jul 2 22:14:14 2020 From: rspamd at shamm.as (George Shammas) Date: Thu, 2 Jul 2020 22:14:14 +0000 Subject: [Rspamd-Users] Multiple dkim selectors without using a map? In-Reply-To: References: <20200630152353.l72x2vfhqul5gowa@shamm.as> Message-ID: <20200702221414.vsq2wyx5j534ld45@shamm.as> Thanks for the pointers. The most interesting in that list is vault. Went on a configuration bender today getting that working. The handling of key rotation is a nice plus. Something that is interesting with rspamd with vault, is that it takes a token from either a configuration file or an environment variable. But the only non-expring token from vault are root tokens. There doesn't seem to be a good method to have rspamd refresh the tokens. To work-around this, I ended up rigging up systemd to get a new token every time it starts. However even in that case, it means that if rspamd is stable for over 32 days (the default max ttl) it will just start silently stop signing mail. Even if I was using a containers to pass down the secret, if the pod was stable it would run into the same problem. Writing a script that produces the map is what I original did, and I'll likely stick to that in the end. It's a bit verbose, but much more bullet proof than the alternatives. --George On Wed, Jul 01, 2020 at 01:55:26PM +0200, Arno Welzel wrote: > > There are several ways to do this, not only maps. > > > > > > > > Another way would be to create a script which produces this map file > based on the list of domain names. > From gabriele.nencioni at register.it Fri Jul 3 14:36:58 2020 From: gabriele.nencioni at register.it (Gabriele Nencioni) Date: Fri, 3 Jul 2020 16:36:58 +0200 Subject: [Rspamd-Users] rspamd per-user statistic user limit Message-ID: Hi, I have upgrade Rspamd from 1.8 to 2.4 version, and I have the following statistic configuration: cat /etc/rspamd/local.d/statistic.conf classifier { bayes { tokenizer { name = "osb"; } name = "global"; backend = 'redis'; read_servers = 'xxx.xxx.xxx.xxx'; write_servers = 'xxx.xxx.xxx.xxx'; timeout = 30s; new_schema = true; expire = 10368000; lazy = true; min_tokens = 11; min_learns = 200; languages_enabled = true; statfile { spam = false; symbol = "BAYES_HAM"; } statfile { spam = true; symbol = "BAYES_SPAM"; } learn_condition = <= 0.95 else cl = 'ham' in_class = prob <= 0.05 end if in_class then return false,string.format('already in class %s; probability %.2f%%', cl, math.abs((prob - 0.5) * 200.0)) end end end return true end EOD; } } classifier { bayes { tokenizer { name = "osb"; } name = "peruser"; backend = 'redis'; read_servers = 'xxx.xxx.xxx.xxx'; write_servers = 'xxx.xxx.xxx.xxx'; timeout = 30s; new_schema = true; expire = 10368000; lazy = true; min_tokens = 11; min_learns = 0; languages_enabled = true; users_enabled = true; per_user = true; statfile { symbol = "BAYES_HAM_USER"; spam = false; } statfile { symbol = "BAYES_SPAM_USER"; spam = true; } learn_condition = <= 0.95 else cl = 'ham' in_class = prob <= 0.05 end if in_class then return false,string.format('already in class %s; probability %.2f%%', cl, math.abs((prob - 0.5) * 200.0)) end end end return true end EOD; } } and my problem is: on peruser classifier every time 1000 users are reached, the learned and users counters are reset, as you can see by rspamc stat output: ~# while date ; do rspamc stat | grep _HAM_USER ; sleep 60 ; done ... Statfile: BAYES_HAM_USER type: redis; length: 78.07M; free blocks: 0; total blocks: 2.05M; free: 0.00%; learned: 3301; users: 924; languages: 0 ... Statfile: BAYES_HAM_USER type: redis; length: 84.27M; free blocks: 0; total blocks: 2.21M; free: 0.00%; learned: 3511; users: 977; languages: 0 ... Statfile: BAYES_HAM_USER type: redis; length: 1.97M; free blocks: 0; total blocks: 51.86k; free: 0.00%; learned: 66; users: 26; languages: 0 while the number of keys on redis isn't decreased. (Same behavior with BAYES_SPAM_USER and with old_schema setting) On Rspamd 1.8 (where the only configuration difference is the old_schema) I have not any problem: ~# rspamc stat | grep _USER Statfile: BAYES_SPAM_USER type: redis; length: 27.55G; free blocks: 0; total blocks: 725M; free: 0.00%; learned: 2246954; users: 44086; languages: 0 Statfile: BAYES_HAM_USER type: redis; length: 9.76G; free blocks: 0; total blocks: 256.83M; free: 0.00%; learned: 561230; users: 33171; languages: 0 Which setting have I to change in order to disable that limit? Thanks in advance Regards, -- Gabriele Nencioni From gabriele.nencioni at register.it Sat Jul 4 08:12:49 2020 From: gabriele.nencioni at register.it (Gabriele Nencioni) Date: Sat, 4 Jul 2020 10:12:49 +0200 Subject: [Rspamd-Users] rspamd per-user statistic user limit In-Reply-To: References: Message-ID: <18361932-f41c-d3e6-866b-59c73dda05a7@register.it> On 7/3/20 4:36 PM, Gabriele Nencioni wrote: > and my problem is: on peruser classifier every time 1000 users are > reached, the learned and users counters are reset, as you can see by > rspamc stat output: Sorry, I have forgot to say I'm trying to perform a massive learning of thousand ham and spam messages from a directory. So how can I do that avoiding the counters have been reset to zero? Maybe have I to do that with a slowly process learning one message at a time with a sleep? Any suggestions will be appreciated... Thank you very much Regards,-- Gabriele Nencioni From gabriele.nencioni at register.it Tue Jul 7 10:03:48 2020 From: gabriele.nencioni at register.it (Gabriele Nencioni) Date: Tue, 7 Jul 2020 12:03:48 +0200 Subject: [Rspamd-Users] rspamd per-user statistic user limit In-Reply-To: <18361932-f41c-d3e6-866b-59c73dda05a7@register.it> References: <18361932-f41c-d3e6-866b-59c73dda05a7@register.it> Message-ID: <8b528e55-128a-2e28-e3eb-946a4a8fdf93@register.it> On 7/4/20 10:12 AM, Gabriele Nencioni wrote: > Sorry, I have forgot to say I'm trying to perform a massive learning of > thousand ham and spam messages from a directory. > > So how can I do that avoiding the counters have been reset to zero? Hi all again, about this problem, after I have performed severl tests, I would add some news. I think it's a statfile counter issue because all the keys are properly stored on redis backend. Moreover using the same redis backend for rspamd 1.8.3 and 2.4.2 servers: ~# rspamd -v Rspamd daemon version 1.8.3 ~# rspamadm configdump | grep _servers write_servers = "172.20.235.61"; read_servers = "172.20.235.62"; ~# rspamd -v Rspamd daemon version 2.4 ~# rspamadm configdump | grep _servers write_servers = "172.20.235.61"; read_servers = "172.20.235.62"; I have the following rspamc stat output result: ~# rspamd -v Rspamd daemon version 1.8.3 ~# rspamc stat ... Statfile: BAYES_SPAM_USER type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 3953; users: 1594; languages: 0 Statfile: BAYES_HAM_USER type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 1948; users: 1321; languages: 0 Total learns: 5901 ~# rspamd -v Rspamd daemon version 2.4 ~# rspamc stat ... Statfile: BAYES_SPAM_USER type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 1535; users: 594; languages: 0 Statfile: BAYES_HAM_USER type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 444; users: 321; languages: 0 Total learns: 1979 so exactly 1000 users less for each statfile. Moreover also the "Total learns" counter is different as you can see from above output. The issue occurs only on peruser classifier. Since the whole Keyspace is ok, the rspamc scan properly works and each message (learned) properly triggered the BAYES_HAM_USER or BAYES_SPAM_USER symbols. So can you confirm it is a statfile counter visualization problem? If so, how can I fix it? Thanks in advance Regards, -- Gabriele Nencioni From ihor at antonovs.family Tue Jul 7 16:12:21 2020 From: ihor at antonovs.family (Ihor Antonov) Date: Tue, 7 Jul 2020 09:12:21 -0700 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists Message-ID: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> Hi, A lot of mailing lists forward emails and thus ruin my DKIM signature. My Inbox fills in with MAILER-DAEMON reports about DKIM signature mismatch when, all of them from recipients who got my message through some mailing list. I wonder if there is an easy way to configure Rspamd to NOT sign outgoing emails if it is being sent to a mailing list? I don't mind maintaining the list of exemptions manually. Thanks! Ihor Antonov From tim at thecutts.org Tue Jul 7 18:23:48 2020 From: tim at thecutts.org (Tim Cutts) Date: Tue, 7 Jul 2020 19:23:48 +0100 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists In-Reply-To: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> References: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> Message-ID: DKIM and SPF are fundamentally fairly hostile to mailing lists. There are some gross hacks to get around it; I run a number of Mailman list servers, and the solution mailman offers is to strip out any incoming DKIM signatures, re-write the headers so that the mail appears to come from the list server itself, and then re-signs with new DKIM headers. That fixes both DKIM and SPF problems from the forwarded emails. In other words, they?re not really forwarded at all, but new emails with fresh signatures. The downside is the recipients can no longer reply direct to the sender of the email. I realise this doesn?t fix your problem, but it?s probably worth educating the list owners if you know them. Regards, Tim > On 7 Jul 2020, at 17:12, Ihor Antonov wrote: > > Hi, > > A lot of mailing lists forward emails and thus ruin my DKIM signature. > My Inbox fills in with MAILER-DAEMON reports about DKIM signature mismatch when, all of them from recipients who got my message through some mailing list. > > I wonder if there is an easy way to configure Rspamd to NOT sign outgoing emails if it is being sent to a mailing list? > I don't mind maintaining the list of exemptions manually. > > Thanks! > > Ihor Antonov > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users From cubewerk at gmail.com Tue Jul 7 19:21:40 2020 From: cubewerk at gmail.com (Stefan Bauer) Date: Tue, 7 Jul 2020 21:21:40 +0200 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists In-Reply-To: References: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> Message-ID: Hi Tim, we did exactly what you've described but still break smime signed mails with thid approach but dont see a solution to this. Thinking if a sender signs mails to ML, he can not expect Stefan Am Dienstag, 7. Juli 2020 schrieb Tim Cutts : > DKIM and SPF are fundamentally fairly hostile to mailing lists. There are > some gross hacks to get around it; I run a number of Mailman list servers, > and the solution mailman offers is to strip out any incoming DKIM > signatures, re-write the headers so that the mail appears to come from the > list server itself, and then re-signs with new DKIM headers. That fixes > both DKIM and SPF problems from the forwarded emails. In other words, > they?re not really forwarded at all, but new emails with fresh signatures. > The downside is the recipients can no longer reply direct to the sender of > the email. > > I realise this doesn?t fix your problem, but it?s probably worth educating > the list owners if you know them. > > Regards, > > Tim > > > On 7 Jul 2020, at 17:12, Ihor Antonov wrote: > > > > Hi, > > > > A lot of mailing lists forward emails and thus ruin my DKIM signature. > > My Inbox fills in with MAILER-DAEMON reports about DKIM signature > mismatch when, all of them from recipients who got my message through some > mailing list. > > > > I wonder if there is an easy way to configure Rspamd to NOT sign > outgoing emails if it is being sent to a mailing list? > > I don't mind maintaining the list of exemptions manually. > > > > Thanks! > > > > Ihor Antonov > > -- > > Users mailing list > > Users at lists.rspamd.com > > https://lists.rspamd.com/mailman/listinfo/users > > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users > From privat at arnowelzel.de Tue Jul 7 21:04:38 2020 From: privat at arnowelzel.de (Arno Welzel) Date: Tue, 7 Jul 2020 23:04:38 +0200 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists In-Reply-To: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> References: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> Message-ID: Ihor Antonov: > A lot of mailing lists forward emails and thus ruin my DKIM signature. > My Inbox fills in with MAILER-DAEMON reports about DKIM signature > mismatch when, all of them from recipients who got my message through > some mailing list. The problem: using DKIM on its own is not really useful without DMARC. Usually you want to combine DKIM with DMARC, so a receiving mail server will not accept mails from your domain without a valid DKIM signature, or at least it will treat those mails as "spammy". So if you omit DKIM signatures for certain mailing lists, you also violate DMARC as spammers would do, when they fake your sender address to get their spam delivered. The real fix is not to avoid DKIM for specific recipients but to ask the mailing list maintainers to set up their lists in a way to deal with it. Also see here: From rspamd at shamm.as Tue Jul 7 22:18:01 2020 From: rspamd at shamm.as (George Shammas) Date: Tue, 7 Jul 2020 22:18:01 +0000 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists In-Reply-To: References: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> Message-ID: <20200707221801.7jlnmqgzh42x2cpl@shamm.as> That article was an interesting read. What does it mean if even this mailing list doesn't deal with this correctly? Just look at the headers of any message from this thread, they all have both dkim and dmarc failures. Here are the auth results for the message you sent. ``` ARC-Authentication-Results: i=1; mail.highsecure.ru; dkim=fail (body hash did not verify) header.d=arnowelzel.de header.s=default header.b=ZGD+5CBZ; spf=none (mail.highsecure.ru: domain of users-bounces at lists.rspamd.com has no SPF policy when checking 10.0.0.75) smtp.mailfrom=users-bounces at lists.rspamd.com ARC-Authentication-Results: i=2; shamm.as; dkim=pass header.d=lists.rspamd.com header.s=dkim header.b=e5VWeO5g; dkim=none (invalid DKIM record) header.d=arnowelzel.de header.s=default header.b=ZGD+5CBZ; arc=pass (lists.rspamd.com:s=arc:i=1); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=arnowelzel.de (policy=none); spf=pass (shamm.as: domain of users-bounces at lists.rspamd.com designates 88.99.142.95 as permitted sender) smtp.mailfrom=users-bounces at lists.rspamd.com ``` On Tue, Jul 07, 2020 at 11:04:38PM +0200, Arno Welzel wrote: > Ihor Antonov: > > > A lot of mailing lists forward emails and thus ruin my DKIM signature. > > My Inbox fills in with MAILER-DAEMON reports about DKIM signature > > mismatch when, all of them from recipients who got my message through > > some mailing list. > > The problem: using DKIM on its own is not really useful without DMARC. > Usually you want to combine DKIM with DMARC, so a receiving mail server > will not accept mails from your domain without a valid DKIM signature, > or at least it will treat those mails as "spammy". > > So if you omit DKIM signatures for certain mailing lists, you also > violate DMARC as spammers would do, when they fake your sender address > to get their spam delivered. > > The real fix is not to avoid DKIM for specific recipients but to ask the > mailing list maintainers to set up their lists in a way to deal with it. > > Also see here: > > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From rspamd at shamm.as Tue Jul 7 23:12:22 2020 From: rspamd at shamm.as (George Shammas) Date: Tue, 7 Jul 2020 23:12:22 +0000 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists In-Reply-To: <20200707221801.7jlnmqgzh42x2cpl@shamm.as> References: <0cc5806e-4a68-afdc-d57e-af0da3aa1b50@antonovs.family> <20200707221801.7jlnmqgzh42x2cpl@shamm.as> Message-ID: <20200707231222.s767d7cci3kfx7j5@shamm.as> More interesting, I can confirm that google groups do correctly deal with dkim signatures and do not have the problem with showing up as failed. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From M.Roos at f1-outsourcing.eu Wed Jul 8 08:01:50 2020 From: M.Roos at f1-outsourcing.eu (Marc Roos) Date: Wed, 8 Jul 2020 10:01:50 +0200 Subject: [Rspamd-Users] Disable DKIM signature for mailing lists In-Reply-To: Message-ID: <"H000007100173ef6.1594195310.sx.f1-outsourcing.eu*"@MHS> Hi Tim, Interesting to read how mailman is offering this service. I was wondering how organisations like ebay are relaying email and prevent people from having direct contact. I tried searching for it a while ago but could not really find something, how are these systems even called? Do you know if it is possible to realize something based on eg. sendmail? -----Original Message----- From: Tim Cutts [mailto:tim at thecutts.org] Sent: dinsdag 7 juli 2020 20:24 To: users at lists.rspamd.com Subject: Re: [Rspamd-Users] Disable DKIM signature for mailing lists DKIM and SPF are fundamentally fairly hostile to mailing lists. There are some gross hacks to get around it; I run a number of Mailman list servers, and the solution mailman offers is to strip out any incoming DKIM signatures, re-write the headers so that the mail appears to come from the list server itself, and then re-signs with new DKIM headers. That fixes both DKIM and SPF problems from the forwarded emails. In other words, they?re not really forwarded at all, but new emails with fresh signatures. The downside is the recipients can no longer reply direct to the sender of the email. I realise this doesn?t fix your problem, but it?s probably worth educating the list owners if you know them. Regards, Tim > On 7 Jul 2020, at 17:12, Ihor Antonov wrote: > > Hi, > > A lot of mailing lists forward emails and thus ruin my DKIM signature. > My Inbox fills in with MAILER-DAEMON reports about DKIM signature mismatch when, all of them from recipients who got my message through some mailing list. > > I wonder if there is an easy way to configure Rspamd to NOT sign outgoing emails if it is being sent to a mailing list? > I don't mind maintaining the list of exemptions manually. > > Thanks! > > Ihor Antonov > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users -- Users mailing list Users at lists.rspamd.com https://lists.rspamd.com/mailman/listinfo/users From gazorg at gmail.com Wed Jul 8 20:06:16 2020 From: gazorg at gmail.com (David Krantz) Date: Wed, 8 Jul 2020 22:06:16 +0200 Subject: [Rspamd-Users] ARC-signing does not happen Message-ID: Hi, I cannot get rspamd to set ARC signatures to my outgoing mails. I have a list server so both incoming and outgoing mails are passed through the milter. Incoming mails should be ARC-verified (if applicable) and outgoing should be signed. I know that the outgoing milter works as it adds DKIM signatures which works fine. If there is any resource on how to do this or if someone has a tip on how to proceed I'd be very happy. I run postfix as MTA and the following config for ARC in rspamd (domain name changed to protect the innocent): /etc/rspamd# rspamadm configdump arc *** Section arc *** use_esld = true; allow_hdrfrom_mismatch = true; selector = "dkim"; key_prefix = "ARC_KEYS"; sign_networks [ "127.0.0.1/8", ] auth_only = false; allow_envfrom_empty = true; try_fallback = true; domain { example.net { path = "/var/rspamd/dkim/example.net.dkim.key"; selector = "dkim"; } } symbol_sign = "ARC_SIGNED"; use_redis = false; allow_username_mismatch = true; sign_local = true; sign_inbound = true; use_domain = "recipient"; allow_hdrfrom_multiple = true; *** End of section arc *** Cheers // David From anthony at mindmedia.com.sg Thu Jul 9 15:24:37 2020 From: anthony at mindmedia.com.sg (P.V.Anthony) Date: Thu, 9 Jul 2020 23:24:37 +0800 Subject: [Rspamd-Users] ARC-signing does not happen In-Reply-To: References: Message-ID: On 9/7/20 04:06, David Krantz wrote: > I cannot get rspamd to set ARC signatures to my outgoing mails. I have > a list server so both incoming and outgoing mails are passed through > the milter. Incoming mails should be ARC-verified (if applicable) and > outgoing should be signed. I know that the outgoing milter works as it > adds DKIM signatures which works fine. If there is any resource on how > to do this or if someone has a tip on how to proceed I'd be very > happy. Please note that I am not an expert. The following seems to work for me. P.V.Anthony /etc/rspamd/local.d/arc.conf # If false, messages with empty envelope from are not signed allow_envfrom_empty = false; # If true, envelope/header domain mismatch is ignored allow_hdrfrom_mismatch = false; # If true, multiple from headers are allowed (but only first is used) allow_hdrfrom_multiple = false; # If true, username does not need to contain matching domain allow_username_mismatch = false; # If false, messages from authenticated users are not selected for signing (needed) auth_only = true; # Default selector to use (needed) selector = "default"; # If false, messages from local networks are not selected for signing sign_local = true; # Symbol to add when message is signed symbol_signed = "ARC_SIGNED"; # Domain to use for ARC signing: can be "header" or "envelope" (needed) use_domain = "header"; # Whether to normalise domains to eSLD use_esld = false; try_fallback = false; path_map = "/etc/rspamd/local.d/dkim_domains.map"; selector_map = "/etc/rspamd/local.d/dkim_selectors.map"; From gazorg at gmail.com Sat Jul 11 10:20:02 2020 From: gazorg at gmail.com (David Krantz) Date: Sat, 11 Jul 2020 12:20:02 +0200 Subject: [Rspamd-Users] ARC-signing does not happen In-Reply-To: References: Message-ID: Hi, Sorry, that does not work either. I skimmed the lua source for the arc module very quickly and I have a hard time seeing how it could start an ARC chain at all. It can continue one, though. I cannot get the module to add any signatures or headers at all. I just get the ARC_NA symbol in the history. Therefore I read parts of the ARC-module source very quickly. As I read it, it only can verify existing ARC chains and cannot start new ones. I have no dev environment setup for this so I link to github's web version here instead: https://github.com/rspamd/rspamd/blob/1fa88bcd56301f2f41319b987ac89909c87b6d0b/src/plugins/lua/arc.lua What I think is the entry to the arc module starts like follows below. So if there are no ARC headers, the function exits and sets the symbol "ARC_NA" on the message. This is not the behaviour I would like. local function arc_callback(task) local arc_sig_headers = task:get_header_full('ARC-Message-Signature') local arc_seal_headers = task:get_header_full('ARC-Seal') if not arc_sig_headers or not arc_seal_headers then task:insert_result(arc_symbols['na'], 1.0) return end As a list server normally is the first recipient in a chain, there normally will be no ARC-headers but as the list forwards the mail ARC headers with i=1 should be added based on the results for DKIM and SPF on the incoming mails. Of course you can add headers on the incoming with i=1 and on the outgoing with i=2, that is maybe even better. You would have to ignore the failed signature check when performing the checks on the outgoing side as you know that the mail will have been changed. Then you know what domains to accept broken signatures from as that is what you signed with (and there is a list of domains to consider as automatically valid in the config, IIRC, so that might work). On the incoming you'd use the recipient domain in the header or envelope and on the outgoing you'd use a different source for the domain depending on how the mail list is set up - the header "To:" or "From:" should be the ones to use in my setup, the point being that it could be different from when signing the incoming mail. I could write a change request for this of course. I can make a change also but as I have too little time it would probably be a solution that worked for my case only. As I do not want to set up a complete rspamd dev environment I guess that the easy way out would be a local LUA-ruleset based on the current ARC module. Cheers // David On Thu, Jul 9, 2020 at 5:25 PM P.V.Anthony wrote: > > On 9/7/20 04:06, David Krantz wrote: > > > I cannot get rspamd to set ARC signatures to my outgoing mails. I have > > a list server so both incoming and outgoing mails are passed through > > the milter. Incoming mails should be ARC-verified (if applicable) and > > outgoing should be signed. I know that the outgoing milter works as it > > adds DKIM signatures which works fine. If there is any resource on how > > to do this or if someone has a tip on how to proceed I'd be very > > happy. > > Please note that I am not an expert. > > The following seems to work for me. > > P.V.Anthony > > /etc/rspamd/local.d/arc.conf > > # If false, messages with empty envelope from are not signed > allow_envfrom_empty = false; > > # If true, envelope/header domain mismatch is ignored > allow_hdrfrom_mismatch = false; > > # If true, multiple from headers are allowed (but only first is used) > allow_hdrfrom_multiple = false; > > # If true, username does not need to contain matching domain > allow_username_mismatch = false; > > # If false, messages from authenticated users are not selected for > signing (needed) > auth_only = true; > > # Default selector to use (needed) > selector = "default"; > > # If false, messages from local networks are not selected for signing > sign_local = true; > > # Symbol to add when message is signed > symbol_signed = "ARC_SIGNED"; > > # Domain to use for ARC signing: can be "header" or "envelope" (needed) > use_domain = "header"; > > # Whether to normalise domains to eSLD > use_esld = false; > > > try_fallback = false; > path_map = "/etc/rspamd/local.d/dkim_domains.map"; > selector_map = "/etc/rspamd/local.d/dkim_selectors.map"; > > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users From vsevolod at rspamd.com Sat Jul 11 10:27:08 2020 From: vsevolod at rspamd.com (Vsevolod Stakhov) Date: Sat, 11 Jul 2020 11:27:08 +0100 Subject: [Rspamd-Users] ARC-signing does not happen In-Reply-To: References: Message-ID: Hello David, Please refrain from top posting here. On 11/07/2020 11:20, David Krantz wrote: > Hi, > > Sorry, that does not work either. > > I skimmed the lua source for the arc module very quickly and I have a > hard time seeing how it could start an ARC chain at all. It can > continue one, though. > > I cannot get the module to add any signatures or headers at all. I > just get the ARC_NA symbol in the history. Therefore I read parts of > the ARC-module source very quickly. As I read it, it only can verify > existing ARC chains and cannot start new ones. I have no dev > environment setup for this so I link to github's web version here > instead: > > https://github.com/rspamd/rspamd/blob/1fa88bcd56301f2f41319b987ac89909c87b6d0b/src/plugins/lua/arc.lua > > What I think is the entry to the arc module starts like follows below. > So if there are no ARC headers, the function exits and sets the symbol > "ARC_NA" on the message. This is not the behaviour I would like. > > local function arc_callback(task) > local arc_sig_headers = task:get_header_full('ARC-Message-Signature') > local arc_seal_headers = task:get_header_full('ARC-Seal') > > if not arc_sig_headers or not arc_seal_headers then > task:insert_result(arc_symbols['na'], 1.0) > return > end > > As a list server normally is the first recipient in a chain, there > normally will be no ARC-headers but as the list forwards the mail ARC > headers with i=1 should be added based on the results for DKIM and SPF > on the incoming mails. Of course you can add headers on the incoming > with i=1 and on the outgoing with i=2, that is maybe even better. You > would have to ignore the failed signature check when performing the > checks on the outgoing side as you know that the mail will have been > changed. Then you know what domains to accept broken signatures from > as that is what you signed with (and there is a list of domains to > consider as automatically valid in the config, IIRC, so that might > work). On the incoming you'd use the recipient domain in the header or > envelope and on the outgoing you'd use a different source for the > domain depending on how the mail list is set up - the header "To:" or > "From:" should be the ones to use in my setup, the point being that it > could be different from when signing the incoming mail. > > I could write a change request for this of course. I can make a change > also but as I have too little time it would probably be a solution > that worked for my case only. As I do not want to set up a complete > rspamd dev environment I guess that the easy way out would be a local > LUA-ruleset based on the current ARC module. > ARC check and ARC sign are different symbols (functions) and they are completely separated (indeed ARC_NA symbol is completely legit here, even if ARC_SIGN has also taken place). The first signature should have cv=none and the following signatures should have cv equal to seals check results. Hence, I have no idea what change do you suggest. From gazorg at gmail.com Sat Jul 11 12:01:27 2020 From: gazorg at gmail.com (David Krantz) Date: Sat, 11 Jul 2020 14:01:27 +0200 Subject: [Rspamd-Users] ARC-signing does not happen In-Reply-To: References: Message-ID: On Sat, Jul 11, 2020 at 12:27 PM Vsevolod Stakhov wrote: > Please refrain from top posting here. My bad, sorry for that. > ARC check and ARC sign are different symbols (functions) and they are > completely separated (indeed ARC_NA symbol is completely legit here, > even if ARC_SIGN has also taken place). The first signature should have > cv=none and the following signatures should have cv equal to seals check > results. Hence, I have no idea what change do you suggest. Okay, missed that function. Too quick read, hence. I still cannot get the arc module to emit any header at all, the only effect I have found from any config is the ARC_NA symbol in the rspamd-log. I have not been successful in activating debug logging for the module, adding "arc" to the debug_modules-list and then reloading rspamd did not give any effect at least. // David From gazorg at gmail.com Sat Jul 11 22:00:38 2020 From: gazorg at gmail.com (David Krantz) Date: Sun, 12 Jul 2020 00:00:38 +0200 Subject: [Rspamd-Users] ARC-signing does not happen In-Reply-To: References: Message-ID: On Sat, Jul 11, 2020 at 2:01 PM David Krantz wrote: > I still cannot get the arc module to emit any header at all, the only > effect I have found from any config is the ARC_NA symbol in the > rspamd-log. I have not been successful in activating debug logging for > the module, adding "arc" to the debug_modules-list and then reloading > rspamd did not give any effect at least. Are there any hidden prerequisites for ARC-signing to work? I think that I have exhausted the available options for configuration, and I have tested the config above and others without any visible effect. The source code for the ARC module has a few dependencies registered but I do not know how that works. My ARC config is as follows: *** Section arc *** use_esld = true; key_prefix = "ARC_KEYS"; allow_envfrom_empty = true; symbol_sign = "ARC_SIGNED"; allow_username_mismatch = true; sign_local = true; use_domain_sign_inbound = "recipient"; allow_hdrfrom_mismatch = true; selector = "dkim"; auth_only = true; try_fallback = false; path = "/var/lib/rspamd/dkim/$domain.$selector.key"; use_redis = false; sign_inbound = true; domain { example.com { path = "/var/lib/rspamd/dkim/example.com.dkim.key"; selector = "dkim"; } } domain { listsrv.example.com { path = "/var/lib/rspamd/dkim/listsrv.example.com.dkim.key"; selector = "dkim"; } } use_domain = "header"; allow_hdrfrom_multiple = true; *** End of section arc *** I have tried with auth_only=false and many other iterations but without any success. So I start to suspect that there is something fundamental missing that I have missed. Happy for any suggestions. Cheers. // David From phils at caerllewys.net Sat Jul 11 22:57:46 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Sat, 11 Jul 2020 18:57:46 -0400 Subject: [Rspamd-Users] Troubleshooting a delivery problem Message-ID: My gmail account redirects to a local account on my own mailserver. It is racking up vast numbers of delivery failure notices, which it tries to forward to me, which bounce causing a delivery failure notice, which it tries to forward to me, which bounces causing a delivery failure notice, which it tries to forward... Delivery is failing with a milter rejection. Example: Jul 11 17:50:54 minbar postfix/cleanup[4796]: 708AB40379A09: milter-reject: END-OF-MESSAGE from mail-lj1-f176.google.com[209.85.208.176]: 5.7.1 Spam message rejected; from= to= proto=ESMTP helo= I narrowed the rejection down to dspam, which is scoring the delivery failure notice 18.88/15, with the following symbols: ARC_REJECT, BAYES_SPAM, DKIM_TRACE, DMARC_POLICY_REJE, DWL_DNSWL_BLOCKED, HAS_REPLYTO, MIME_BASE64_TEXT, MIME_MA_MISSING_TEXT, R_DKIM_REJECT, R_SPF_ALLOW, RCVD_COUNT_THREE, REPLYTO_DN_EQ_FROM_DN, SEM_URIBL_FRESH15_UNKNOWN_FAIL, TO_DN_NONE, URI_COUNT_ODD I figure I can fix this with a whitelist override entry. But what I really want to whitelist is messages from googlemail.com or google.com containing the string "There was a problem delivering your message to". How can I add that string to a whitelist ruleset? For that matter, what's the correct syntax for a $LOCAL_CONFDIR/override.d/whitelist.conf file? Nothing I've tried works. If I include an outer 'whitelist { }' as the documentation seems to say, it is rejected because of a nested whitelist. If I don't, I get errors telling me that symbols WHITELIST_SPF, BLACKLIST_DMARC etc have scores defined but no corresponding rule registered. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From rspamd at shamm.as Sun Jul 12 03:45:30 2020 From: rspamd at shamm.as (George Shammas) Date: Sun, 12 Jul 2020 03:45:30 +0000 Subject: [Rspamd-Users] Spamtrap module looks only at To header and not actual recipient Message-ID: <20200712034530.2ig5juwxmqbx4dbu@shamm.as> I'm a big fan of wildcard domains and setting up aliases for all services I use. If a service get's compromised it can be bucketed towards spam. There are now several of these emails that get a significant amount of spam, but only about a third actually have a To: header at all. The rest either have spoofed To: are missing To: headers. The spamtrap module seems to only look at the To: header and so they all squeak by the spamtrap rules. There doesn't seem to be a method to make the spamtrap module look at who the mail was originally intended for. And since spammers are so great at spoofing headers it also greatly lowers the effectiveness of the module. Is there a way to make the spamtrap module look at the recipient instead of the To: header? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From moiseev at mezonplus.ru Sun Jul 12 06:40:32 2020 From: moiseev at mezonplus.ru (Alexander Moisseev) Date: Sun, 12 Jul 2020 09:40:32 +0300 Subject: [Rspamd-Users] Spamtrap module looks only at To header and not actual recipient In-Reply-To: <20200712034530.2ig5juwxmqbx4dbu@shamm.as> References: <20200712034530.2ig5juwxmqbx4dbu@shamm.as> Message-ID: <57652e65-471c-87bb-c9fc-f2dcdf3665f6@mezonplus.ru> 12.07.2020 6:45, George Shammas via Users ?????: > I'm a big fan of wildcard domains and setting up aliases for all > services I use. If a service get's compromised it can be bucketed > towards spam. > > There are now several of these emails that get a significant amount of > spam, but only about a third actually have a To: header at all. The > rest either have spoofed To: are missing To: headers. The spamtrap > module seems to only look at the To: header and so they all squeak by > the spamtrap rules. > > There doesn't seem to be a method to make the spamtrap module look at > who the mail was originally intended for. And since spammers are so > great at spoofing headers it also greatly lowers the effectiveness of > the module. > > Is there a way to make the spamtrap module look at the recipient > instead of the To: header? > The spamtrap module doesn't look at the To: header, it checks SMTP recipients. https://github.com/rspamd/rspamd/blob/1b66df81cd9fa6799f1fd8d2fd92781661d722d8/src/plugins/lua/spamtrap.lua#L40 From gazorg at gmail.com Sun Jul 12 10:05:36 2020 From: gazorg at gmail.com (David Krantz) Date: Sun, 12 Jul 2020 12:05:36 +0200 Subject: [Rspamd-Users] ARC-signing does not happen In-Reply-To: References: Message-ID: On Sun, Jul 12, 2020 at 12:00 AM David Krantz wrote: > I have tried with auth_only=false and many other iterations but > without any success. So I start to suspect that there is something > fundamental missing that I have missed. Happy for any suggestions. And now I found the error. It was a stale rspamd process that prevented reloads of config in combination with sloppy reading of the rspamd.log from my part. Sorry for that. // David From rspamd at shamm.as Sun Jul 12 12:07:27 2020 From: rspamd at shamm.as (George Shammas) Date: Sun, 12 Jul 2020 12:07:27 +0000 Subject: [Rspamd-Users] Spamtrap module looks only at To header and not actual recipient In-Reply-To: <57652e65-471c-87bb-c9fc-f2dcdf3665f6@mezonplus.ru> References: <20200712034530.2ig5juwxmqbx4dbu@shamm.as> <57652e65-471c-87bb-c9fc-f2dcdf3665f6@mezonplus.ru> Message-ID: <20200712120727.6hresjuxtvykwjbt@shamm.as> On Sun, Jul 12, 2020 at 09:40:32AM +0300, Alexander Moisseev via Users wrote: > 12.07.2020 6:45, George Shammas via Users ?????: > > Is there a way to make the spamtrap module look at the recipient > > instead of the To: header? > > > > The spamtrap module doesn't look at the To: header, it checks SMTP recipients. > > https://github.com/rspamd/rspamd/blob/1b66df81cd9fa6799f1fd8d2fd92781661d722d8/src/plugins/lua/spamtrap.lua#L40 The code does suggest that you are correct... Dug a little deeper, there was only a few lines in my spamtrap map, and the one with excessive spoofing was the last line. That line was missing the trailing newline, rendering it invalid. Sorry for the confusion. From gabriele.nencioni at register.it Mon Jul 13 06:28:29 2020 From: gabriele.nencioni at register.it (Gabriele Nencioni) Date: Mon, 13 Jul 2020 08:28:29 +0200 Subject: [Rspamd-Users] rspamd per-user statistic user limit In-Reply-To: <8b528e55-128a-2e28-e3eb-946a4a8fdf93@register.it> References: <18361932-f41c-d3e6-866b-59c73dda05a7@register.it> <8b528e55-128a-2e28-e3eb-946a4a8fdf93@register.it> Message-ID: Hi, line 942 of src/libstat/backends/redis_backend.c source file: /* Get more stat keys */ redisAsyncCommand (cbdata->redis, rspamd_redis_stat_keys, redis_elt, "SSCAN %s_keys %s COUNT 1000", cbdata->elt->ctx->stcf->symbol, so are the statistic limited by design? Regards, -- Gabriele Nencioni From eduard at martinescu.org Mon Jul 13 18:32:03 2020 From: eduard at martinescu.org (Eduard Martinescu) Date: Mon, 13 Jul 2020 14:32:03 -0400 Subject: [Rspamd-Users] DMARC module report configuration Message-ID: <34F9FA68-6E81-493D-8D68-BD93A059DDF6@martinescu.org> Are there any options to specify authentication information for the attempt to send the DMARC report that gets generated? I see settings available to set the host and port, but not to provide any credentials to authenticate. Or do I have to just whitelist the rspamd host? Thanks Ed Martinescu From ganesh.malyala at gmail.com Tue Jul 14 10:52:42 2020 From: ganesh.malyala at gmail.com (Venkata Ganesh Raju Malyala) Date: Tue, 14 Jul 2020 16:22:42 +0530 Subject: [Rspamd-Users] Regarding Malicious File blocking using hashes in rspamd Message-ID: Hello everyone, Can someone advise blocking malicious files/ attachments in rspamd if we have a set of malicious file hashes. Thank you *Ganesh* From riccardo.alfieri at spamteq.com Tue Jul 14 11:29:26 2020 From: riccardo.alfieri at spamteq.com (Riccardo Alfieri) Date: Tue, 14 Jul 2020 11:29:26 +0000 Subject: [Rspamd-Users] Regarding Malicious File blocking using hashes in rspamd In-Reply-To: References: Message-ID: On 14/07/20 12:52, Venkata Ganesh Raju Malyala wrote: > Hello everyone, > > Can someone advise blocking malicious files/ attachments in rspamd if we > have a set of malicious file hashes. > Hi, you can take inspiration from the function called "check_file_callback" here: https://github.com/spamhaus/rspamd-dqs/blob/master/2.x/rspamd.local.lua#L97 Depending on the type of the hash you have you'll probably need to adjust the "local filehash" line #103 -- Best regards, Riccardo Alfieri Spamhaus Technology https://www.spamhaustech.com/ From vsevolod at rspamd.com Tue Jul 14 11:51:32 2020 From: vsevolod at rspamd.com (Vsevolod Stakhov) Date: Tue, 14 Jul 2020 12:51:32 +0100 Subject: [Rspamd-Users] Regarding Malicious File blocking using hashes in rspamd In-Reply-To: References: Message-ID: <798dc038-b87e-c889-12d6-1fd2dfdaa5c3@rspamd.com> On 14/07/2020 12:29, Riccardo Alfieri wrote: > On 14/07/20 12:52, Venkata Ganesh Raju Malyala wrote: > >> Hello everyone, >> >> Can someone advise blocking malicious files/ attachments in rspamd if we >> have a set of malicious file hashes. >> > Hi, > > you can take inspiration from the function called "check_file_callback" > here: > https://github.com/spamhaus/rspamd-dqs/blob/master/2.x/rspamd.local.lua#L97 > > Depending on the type of the hash you have you'll probably need to > adjust the "local filehash" line #103 > As I have said, this is a wrong approach as it is alien to Rspamd architecture and requires lot's of extra work (e.g. multiple sha256 calculations). Here is the correct one: spamhaus_dqs_hbl { symbol = "HBL_FILE_UNKNOWN"; rbl = "_file.{= SPAMHAUS_DQS_KEY =}.hbl.dq.spamhaus.net."; selector = "attachments('rbase32', 'sha256')"; ignore_whitelist = true; ignore_defaults = true; returncodes { HBL_FILE_MALICIOUS = "127.0.3.10"; HBL_FILE_SUSPICIOUS = "127.0.3.15"; } } Then you can specify SPAMHAUS_DQS_KEY via environment when you start Rspamd to avoid key leak via configuration. This works with 2.6 only, however. From riccardo.alfieri at spamteq.com Tue Jul 14 13:12:26 2020 From: riccardo.alfieri at spamteq.com (Riccardo Alfieri) Date: Tue, 14 Jul 2020 13:12:26 +0000 Subject: [Rspamd-Users] Regarding Malicious File blocking using hashes in rspamd In-Reply-To: <798dc038-b87e-c889-12d6-1fd2dfdaa5c3@rspamd.com> References: <798dc038-b87e-c889-12d6-1fd2dfdaa5c3@rspamd.com> Message-ID: On 14/07/20 13:51, Vsevolod Stakhov wrote: > As I have said, this is a wrong approach as it is alien to Rspamd > architecture and requires lot's of extra work (e.g. multiple sha256 > calculations). Here is the correct one: > > spamhaus_dqs_hbl { > symbol = "HBL_FILE_UNKNOWN"; > rbl = "_file.{= SPAMHAUS_DQS_KEY =}.hbl.dq.spamhaus.net."; > selector = "attachments('rbase32', 'sha256')"; > ignore_whitelist = true; > ignore_defaults = true; > returncodes { > HBL_FILE_MALICIOUS = "127.0.3.10"; > HBL_FILE_SUSPICIOUS = "127.0.3.15"; > } > } Thank you very much for pointing out to me how selectors work. Is there a way I can check the Rspamd version through rspamd.local.lua? In this way I could possibly dynamically use RFC base32 feature if Rspamd>=2.6 and keep using the "old" approach for versions 2.0-2.5 -- Best regards, Riccardo Alfieri Spamhaus Technology https://www.spamhaustech.com/ From vsevolod at rspamd.com Tue Jul 14 13:42:16 2020 From: vsevolod at rspamd.com (Vsevolod Stakhov) Date: Tue, 14 Jul 2020 14:42:16 +0100 Subject: [Rspamd-Users] Regarding Malicious File blocking using hashes in rspamd In-Reply-To: References: <798dc038-b87e-c889-12d6-1fd2dfdaa5c3@rspamd.com> Message-ID: <80f0448c-49be-ffab-0011-ebb3b1aafd91@rspamd.com> On 14/07/2020 14:12, Riccardo Alfieri wrote: > On 14/07/20 13:51, Vsevolod Stakhov wrote: > >> As I have said, this is a wrong approach as it is alien to Rspamd >> architecture and requires lot's of extra work (e.g. multiple sha256 >> calculations). Here is the correct one: >> >> spamhaus_dqs_hbl { >> ?? symbol = "HBL_FILE_UNKNOWN"; >> ?? rbl = "_file.{= SPAMHAUS_DQS_KEY =}.hbl.dq.spamhaus.net."; >> ?? selector = "attachments('rbase32', 'sha256')"; >> ?? ignore_whitelist = true; >> ?? ignore_defaults = true; >> ?? returncodes { >> ???? HBL_FILE_MALICIOUS = "127.0.3.10"; >> ???? HBL_FILE_SUSPICIOUS = "127.0.3.15"; >> ?? } >> } > Thank you very much for pointing out to me how selectors work. Is there > a way I can check the Rspamd version through rspamd.local.lua? In this > way I could possibly dynamically use RFC base32 feature if Rspamd>=2.6 > and keep using the "old" approach for versions 2.0-2.5 > You can use jinja templates in all Rspamd config files: {% if tonumber(version or '0') >= 2.6 %} # Config for 2.6+ {% else %} # Legacy config {% endif %} From vsevolod at rspamd.com Tue Jul 14 13:45:52 2020 From: vsevolod at rspamd.com (Vsevolod Stakhov) Date: Tue, 14 Jul 2020 14:45:52 +0100 Subject: [Rspamd-Users] Regarding Malicious File blocking using hashes in rspamd In-Reply-To: <80f0448c-49be-ffab-0011-ebb3b1aafd91@rspamd.com> References: <798dc038-b87e-c889-12d6-1fd2dfdaa5c3@rspamd.com> <80f0448c-49be-ffab-0011-ebb3b1aafd91@rspamd.com> Message-ID: <924e5cd4-c376-6183-5b7c-57f1054517ed@rspamd.com> On 14/07/2020 14:42, Vsevolod Stakhov wrote: > On 14/07/2020 14:12, Riccardo Alfieri wrote: >> On 14/07/20 13:51, Vsevolod Stakhov wrote: >> >>> As I have said, this is a wrong approach as it is alien to Rspamd >>> architecture and requires lot's of extra work (e.g. multiple sha256 >>> calculations). Here is the correct one: >>> >>> spamhaus_dqs_hbl { >>> ?? symbol = "HBL_FILE_UNKNOWN"; >>> ?? rbl = "_file.{= SPAMHAUS_DQS_KEY =}.hbl.dq.spamhaus.net."; >>> ?? selector = "attachments('rbase32', 'sha256')"; >>> ?? ignore_whitelist = true; >>> ?? ignore_defaults = true; >>> ?? returncodes { >>> ???? HBL_FILE_MALICIOUS = "127.0.3.10"; >>> ???? HBL_FILE_SUSPICIOUS = "127.0.3.15"; >>> ?? } >>> } >> Thank you very much for pointing out to me how selectors work. Is there >> a way I can check the Rspamd version through rspamd.local.lua? In this >> way I could possibly dynamically use RFC base32 feature if Rspamd>=2.6 >> and keep using the "old" approach for versions 2.0-2.5 >> > > You can use jinja templates in all Rspamd config files: > > {% if tonumber(version or '0') >= 2.6 %} > # Config for 2.6+ > {% else %} > # Legacy config > {% endif %} > One more feature about environment vars: > env RSPAMD_FOO=BAR rspamadm lua LuaJIT 2.0.5> rspamd_env {[ver_major] = 2, [version] = 2.6, [FOO] = BAR, [ver_minor] = 6, [ver_num] = 569547023187968, [hostname] = MacBook-Pro.local, [ver_id] = release} All keys from `rspamd_env` are immediately available in jinja templates. Please bear in mind that `RSPAMD_` prefix that is designed to filter sensitive variables specific to Rspamd only. This prefix is also got removed in `rspamd_env` table. From rspamd at 6x13.de Wed Jul 15 11:43:07 2020 From: rspamd at 6x13.de (Oliver) Date: Wed, 15 Jul 2020 13:43:07 +0200 Subject: [Rspamd-Users] Virus Reject Message Message-ID: <7fadeaa3-3077-60dc-902f-2ac0830d2c37@6x13.de> Hello List, I have a little problem which I can solve by myself. I have configured rspamd to use clamav to scan emails for virus. This works fine, but if I test it and send an E-Mail with an eicar test virus inside rspamd/postfix replys with "5.7.1 Spam message rejected" and I don't know why. I have configured in local.d/antivirus.conf the following line: "message = '${SCANNER}: virus found: "${VIRUS}"';" In my opinion this should indroduce rspamd to reply with a "virus found" message if clamav found a virus. I'm working with rspamd v. 2.5 and ClamAV 0.102.3 on Ubuntu 18.04.4 LTS. If you need more information please tell me. Can anyone help me to solve this? Regards, Oliver From phils at caerllewys.net Wed Jul 15 13:12:13 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Wed, 15 Jul 2020 09:12:13 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? Message-ID: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> Can anyone give me a working example of a /etc/rspamd/override.d/whitelist.conf file please? I an unable to devise a working override file working from the documentation. Everything I have tried throws errors. Also, is it possible to whitelist based on body text? I'm trying to resolve a failure with delivery failure notifications from Gmail being erroneously marked as spam, which cause more delivery failure notifications, which are marked as spam, which cause more delivery failure notifications... -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From rspamd-mlist-user-qa at eminovic.ba Wed Jul 15 19:05:50 2020 From: rspamd-mlist-user-qa at eminovic.ba (=?utf-8?Q?Nihad_=40_RSPAMD_maillist?=) Date: Wed, 15 Jul 2020 21:05:50 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> Message-ID: <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> Working file? That file is not existing as default. You create it, when you have need to "override" modules.d version of the file. So you can actually safely delete that file, and if all is well, functionality should return to normal. Given that you do not have whitelist.conf in the local.d directory either. Here you have original modules.d/whitelist.conf file perhaps that can point you into the right direction. https://github.com/rspamd/rspamd/blob/39be9db453245cf561148e3f0e17c58dec2ce747/conf/modules.d/whitelist.conf As to see what you need to adjust so certain mails do not get marked as spam, you will need to investigate what symbols or what triggers the score. Than you can create an override for that symbol in multimap (i always use local.d/multimap.conf) /Nihad On 15 Jul 2020, 15.13 +0200, User questions , wrote: > > Can anyone give me a working example of a > /etc/rspamd/override.d/whitelist.conf file please? From phils at caerllewys.net Thu Jul 16 02:56:26 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Wed, 15 Jul 2020 22:56:26 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> Message-ID: <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> On 2020-07-15 15:05, Nihad @ RSPAMD maillist via Users wrote: > Working file? > That file is not existing as default. > You create it, when you have need to "override" modules.d version of the file. Yes, that's my point. I'm trying to create an override file to whitelist these delivery failure messages. Perhaps I've been misunderstanding ? I thought that the override file *supplemented* the original file and contained additional stanzas to be added. Is it used *INSTEAD* if found? So I would need to copy the original file into override.d and then add my changes to it, *NOT* simply supply the additional rules I want? > As to see what you need to adjust so certain mails do not get marked as spam, you will need to investigate what symbols or what triggers the score. > Than you can create an override for that symbol in multimap (i always use local.d/multimap.conf) There's quite a few symbols triggered, including BAYES_SPAM, which I suspect means rspamd has Bayes-"learned" that delivery failure notifications are spam. I need to tell it that they are not. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From ml+rspamd at valo.at Thu Jul 16 06:12:59 2020 From: ml+rspamd at valo.at (Christian Kivalo) Date: Thu, 16 Jul 2020 08:12:59 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> Message-ID: On 2020-07-16 04:56, Phil Stracchino wrote: > On 2020-07-15 15:05, Nihad @ RSPAMD maillist via Users wrote: >> Working file? >> That file is not existing as default. >> You create it, when you have need to "override" modules.d version of >> the file. > > > Yes, that's my point. I'm trying to create an override file to > whitelist these delivery failure messages. > > Perhaps I've been misunderstanding ? I thought that the override file > *supplemented* the original file and contained additional stanzas to be > added. Is it used *INSTEAD* if found? So I would need to copy the > original file into override.d and then add my changes to it, *NOT* > simply supply the additional rules I want? Put your supplemental config into local.d See also https://rspamd.com/doc/configuration/ where this is described >> As to see what you need to adjust so certain mails do not get marked >> as spam, you will need to investigate what symbols or what triggers >> the score. >> Than you can create an override for that symbol in multimap (i always >> use local.d/multimap.conf) > > There's quite a few symbols triggered, including BAYES_SPAM, which I > suspect means rspamd has Bayes-"learned" that delivery failure > notifications are spam. I need to tell it that they are not. > > > > -- > Phil Stracchino > Babylon Communications > phils at caerllewys.net > phil at co.ordinate.org > Landline: +1.603.293.8485 > Mobile: +1.603.998.6958 -- Christian Kivalo From rspamd-mlist-user-qa at eminovic.ba Thu Jul 16 11:13:20 2020 From: rspamd-mlist-user-qa at eminovic.ba (=?utf-8?Q?Nihad_=40_RSPAMD_maillist?=) Date: Thu, 16 Jul 2020 13:13:20 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> Message-ID: <9d41e46c-a35a-4cf3-a7f5-db20ef1fe670@Spark> You got it correctly. Config files are ordered and read in a certain order. When you specify something, it will override previous settings. Override.d is loaded last, and hence will override all previous setting when specified in that file. It does not need to be complete copy, just a block og certain setting. If you wish to change score to certain symbols, this is how I do it. Below map is looking for specified domains in a map, and changes their score to -12.0. So if they got score that will mark them as spam, I subtract that score with -12. It usually lowers the score below spam treshold. local.d/multimap.conf [...] WHITELIST_SENDER_DOMAIN { ?type = "header"; ?header = "from"; ?filter = "email:domain"; ?map = "/etc/rspamd/local.d/whitelist.sender.domain.map"; ?score = -12.0; ?regexp = true; } [...] Not tested, but you might be able to add first part of email address, into the map above, and have this work as is. so let?s say gmail sends an "mailbox not found" from an email called daemon-refuse at gmail.com you can specify /daemon-refuse/ in a map, and it will lower score of these mails. You can read a bit more about multimap module and you can look body entries as well. Read more here?https://rspamd.com/doc/modules/multimap.html /Nihad On 16 Jul 2020, 05.03 +0200, User questions , wrote: > > Perhaps I've been misunderstanding ? I thought that the override file > *supplemented* the original file and contained additional stanzas to be > added. Is it used *INSTEAD* if found? So I would need to copy the > original file into override.d and then add my changes to it, *NOT* > simply supply the additional rules I want? From a.seibel at fis-asp.de Thu Jul 16 11:17:33 2020 From: a.seibel at fis-asp.de (Seibel Andreas) Date: Thu, 16 Jul 2020 11:17:33 +0000 Subject: [Rspamd-Users] understanding scoring in multimaps Message-ID: Hello, I have a problem understanding the scoring in multimaps. My local.d/multimap.conf: WHITELIST_FROM_AND_TO { type = "combined"; symbol = "EXCEPTION_FOR_BOB"; score = -5; rules { to = { map = "$LOCAL_CONFDIR/local.d/rcpt_whitelist.map.inc"; selector = "to"; } from = { map = "$LOCAL_CONFDIR/local.d/sender_whitelist.map.inc"; selector = "from"; } ip = { type = "radix"; map = "$LOCAL_CONFDIR/local.d/ip_whitelist.map.inc"; selector = "ip"; } } # expression = "from & to & ip" expression = "to & from" # expression = "to & ip" # expression = "from & ip" } For the expressions "from & to & ip", "to & ip" and "from & ip" it works as expected, but not for "to & from". In the case "to & from" symbol "EXCEPTION_FOR_BOB" is set but score is "-0.00" instead of "-5.00". Example for expression "from & to & ip" (works as expected): # rspamc --ip 10.10.10.10 --from alice at sender.dom --rcpt bob at recipient.dom < ~/testmail.eml Results for file: stdin (0.059 seconds) [Metric: default] Action: no action Spam: false Score: -4.10 / 15.00 Symbol: ARC_NA (0.00) Symbol: EXCEPTION_FOR_BOB (-5.00)[from=alice at sender.dom, to=bob at recipient.dom, ip=10.10.10.10] Symbol: FROM_EQ_ENVFROM (0.00) Symbol: FROM_HAS_DN (0.00) Symbol: MID_RHS_NOT_FQDN (0.50) Symbol: MIME_GOOD (-0.10)[multipart/alternative, text/plain] Symbol: MIME_TRACE (0.00)[0:+, 1:+, 2:~] Symbol: RCPT_COUNT_ONE (0.00)[1] Symbol: RCVD_COUNT_ZERO (0.00)[0] Symbol: R_PARTS_DIFFER (0.50)[100.0%] Symbol: TO_DN_NONE (0.00) Symbol: TO_MATCH_ENVRCPT_ALL (0.00) Message-ID: test-mail at localhost Example for expression "to & from" (symbol is there but score is "-0.00" instead of "-5.00"): # rspamc --ip 10.10.10.10 --from alice at sender.dom --rcpt bob at recipient.dom < ~/testmail.eml Results for file: stdin (0.035 seconds) [Metric: default] Action: no action Spam: false Score: 0.90 / 15.00 Symbol: ARC_NA (0.00) Symbol: EXCEPTION_FOR_BOB (-0.00)[to=bob at recipient.dom, from=alice at sender.dom] Symbol: FROM_EQ_ENVFROM (0.00) Symbol: FROM_HAS_DN (0.00) Symbol: MID_RHS_NOT_FQDN (0.50) Symbol: MIME_GOOD (-0.10)[multipart/alternative, text/plain] Symbol: MIME_TRACE (0.00)[0:+, 1:+, 2:~] Symbol: RCPT_COUNT_ONE (0.00)[1] Symbol: RCVD_COUNT_ZERO (0.00)[0] Symbol: R_PARTS_DIFFER (0.50)[100.0%] Symbol: TO_DN_NONE (0.00) Symbol: TO_MATCH_ENVRCPT_ALL (0.00) Message-ID: test-mail at localhost Does anyone have any idea what I'm doing wrong? Thank you. Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 831 bytes Desc: OpenPGP digital signature URL: From phils at caerllewys.net Thu Jul 16 20:58:54 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Thu, 16 Jul 2020 16:58:54 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> Message-ID: On 2020-07-16 02:12, Christian Kivalo wrote: > Put your supplemental config into local.d > > See also https://rspamd.com/doc/configuration/ where this is described After reading that page for about the fifth time and getting no helpful information from it, and following links to many other documentation pages from many configuration files, I THINK that I MIGHT have figured out what I need. Here's what I have: /etc/rspam/local.d/trie.conf DELIVERY_FAIL_NOTICE { patterns = [ "There was a problem delivering your message", ] } /etc/rspam/local.d/content_group.conf symbols = { "DELIVERY_FAIL_NOTICE" { weight = -5.0; description = "This message looks like a delivery failure notification"; one_shot = true; } } This at least generates no errors on loading. Does this seem like it SHOULD work? Now I get to wait for the next retry and see whether it actually *does* what I want... Am I defining the symbol in the right place? Is there a better place? -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From phils at caerllewys.net Thu Jul 16 23:47:40 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Thu, 16 Jul 2020 19:47:40 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> Message-ID: On 2020-07-16 16:58, Phil Stracchino wrote: > After reading that page for about the fifth time and getting no helpful > information from it, and following links to many other documentation > pages from many configuration files, I THINK that I MIGHT have figured > out what I need. Here's what I have: > > /etc/rspam/local.d/trie.conf > > DELIVERY_FAIL_NOTICE { > patterns = [ > "There was a problem delivering your message", > ] > } > > /etc/rspam/local.d/content_group.conf > > symbols = { > "DELIVERY_FAIL_NOTICE" { > weight = -5.0; > description = "This message looks like a delivery failure > notification"; > one_shot = true; > } > } > > > This at least generates no errors on loading. Does this seem like it > SHOULD work? Now I get to wait for the next retry and see whether it > actually *does* what I want... > > Am I defining the symbol in the right place? Is there a better place? OK, that did not work, my rule apparently did not trigger. Can anyone advise me what I am doing wrong here? -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From rspamd-mlist-user-qa at eminovic.ba Fri Jul 17 08:13:44 2020 From: rspamd-mlist-user-qa at eminovic.ba (=?utf-8?Q?Nihad_=40_RSPAMD_maillist?=) Date: Fri, 17 Jul 2020 10:13:44 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> Message-ID: Can you try this... Create/edit "local.d/groups.conf" file and add following; symbols { "DELIVERY_FAIL_NOTICE" { weight = -40.0; # Define your weight } } You can read more about metrics here, where this example is from, with small adjustment to suit your symbol trigger. https://rspamd.com/doc/configuration/metrics.html /Nihad On 17 Jul 2020, 01.48 +0200, User questions , wrote: > > Can anyone advise me what I am doing wrong here? From gazorg at gmail.com Fri Jul 17 11:37:41 2020 From: gazorg at gmail.com (David Krantz) Date: Fri, 17 Jul 2020 13:37:41 +0200 Subject: [Rspamd-Users] ARC whitelist Message-ID: Hi, How is the whitelisted_signers_map directive in the arc.conf intended to work? I have tried putting a filename in it and adding the whitelisted domains one per line in that file. The file should be readable by rspamd. I still get failures in the verification on ARC signatures from those domains. My setup is a mail list server where the messages should be signed when incoming to verify DKIM and SPF from the original sender and then again after forwarding the mail via a list as the message is altered with a prefix to the subject and a footer. I get two signatures (i=1 with cv=none and i=2 with cv=fail) but I have yet to succeed in getting other than cv=fail on the second signature as the ARC plugin detects changes. I know that the changes are benign so I'd like to sign them as okay although the hash obviously will be broken but I do not get that to work. *** Section arc *** use_esld = true; key_prefix = "ARC_KEYS"; allow_envfrom_empty = true; sign_networks [ "127.0.0.1", ] sign_authenticated = true; symbol_sign = "ARC_SIGNED"; allow_username_mismatch = true; sign_local = true; use_domain_sign_inbound = "recipient"; allow_hdrfrom_mismatch = true; selector = "dkim"; auth_only = true; whitelisted_signers_map = "/etc/rspamd/arc_whitelisted_signers.map"; try_fallback = false; path = "/var/lib/rspamd/dkim/$domain.$selector.key"; domain { listsrv.example.com { path = "/var/lib/rspamd/dkim/listsrv.example.com.dkim.key"; selector = "dkim"; } } domain { example.com { path = "/var/lib/rspamd/dkim/example.com.dkim.key"; selector = "dkim"; } } use_redis = false; reuse_auth_results = true; sign_inbound = true; selector_map = "/etc/rspamd/dkim_selectors.map"; use_domain = "header"; allow_hdrfrom_multiple = true; *** End of section arc *** my /etc/rspamd/arc_whitelisted_signers.map just contains example.com listsrv.example.com cheers // David From phils at caerllewys.net Fri Jul 17 13:39:01 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Fri, 17 Jul 2020 09:39:01 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> Message-ID: <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> On 2020-07-17 04:13, Nihad @ RSPAMD maillist via Users wrote: > Can you try this... > Create/edit "local.d/groups.conf" file and add following; > > symbols { > "DELIVERY_FAIL_NOTICE" { > weight = -40.0; # Define your weight > } > } > > You can read more about metrics here, where this example is from, with small adjustment to suit your symbol trigger. > https://rspamd.com/doc/configuration/metrics.html > > /Nihad Thanks Nihad. I've made that change, now I need to wait for Gmail's next retry to see whether it works. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From phils at caerllewys.net Fri Jul 17 17:40:07 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Fri, 17 Jul 2020 13:40:07 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> Message-ID: On 2020-07-17 09:39, Phil Stracchino wrote: > On 2020-07-17 04:13, Nihad @ RSPAMD maillist via Users wrote: >> Can you try this... >> Create/edit "local.d/groups.conf" file and add following; >> >> symbols { >> "DELIVERY_FAIL_NOTICE" { >> weight = -40.0; # Define your weight >> } >> } >> >> You can read more about metrics here, where this example is from, with small adjustment to suit your symbol trigger. >> https://rspamd.com/doc/configuration/metrics.html >> >> /Nihad > > > > Thanks Nihad. I've made that change, now I need to wait for Gmail's > next retry to see whether it works. Hm, well, that isn't working either, and the DELIVERY_FAIL_NOTICE symbol/rule is not triggering. Could I have missed something necessary to activate the trie module? Should I perhaps be trying to do this via a Subject: header check instead (and if so, how should I do that)? -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From rspamd-mlist-user-qa at eminovic.ba Fri Jul 17 19:03:42 2020 From: rspamd-mlist-user-qa at eminovic.ba (=?utf-8?Q?Nihad_=40_RSPAMD_maillist?=) Date: Fri, 17 Jul 2020 21:03:42 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> Message-ID: The part I gave you is not using trie module. It is just simple symbol score override. If symbol is triggered, it will lower its score by -40. Making accumulated score much lower for messages that rspamd??sees as delivery failure. ?It if that symbol is not triggered you should look at what symbol is triggered and make changes accordingly. Can you give us a list of the symbols that are affecting message score so it scores as spam? I am not sure if true module is right way of doing what you wish to do. Or perhaps I am not understanding what you need. You can do multimap search that can match certain parts of the message. And it even supports regex. So you can have a list of addresses or parts of messages that are triggered with one Multimap definition. This is something I have used with great success to adjust scoring of certain badly configured mail servers. Multimap that inserts?WHITELIST_DELIVERY_FAIL symbol with score of -40 for messages that match user part (user at domain.tld) on messages. in multimap.conf WHITELIST_DELIVERY_FAIL { ?type = "header"; ?header = "from"; ?filter = "email:user"; ?map = "/etc/rspamd/local.d/whitelist_delivery_fail.map"; ?score = -40.0; ?regexp = true; } in whitelist_delivery_fail.map /daemon-failure/ Not tested, but should work, if my theory is correct. :D you just need to populate whitelist_delivery_fail.map with valid strings that emails are coming from. /Nihad On 17 Jul 2020, 19.41 +0200, User questions , wrote: > > Should I perhaps be trying to do this via a Subject: header check > instead (and if so, how should I do that)? From phils at caerllewys.net Fri Jul 17 19:14:25 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Fri, 17 Jul 2020 15:14:25 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> Message-ID: <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> On 2020-07-17 15:03, Nihad @ RSPAMD maillist via Users wrote: > You can do multimap search that can match certain parts of the message. And it even supports regex. > So you can have a list of addresses or parts of messages that are triggered with one Multimap definition. > This is something I have used with great success to adjust scoring of certain badly configured mail servers. > > Multimap that inserts?WHITELIST_DELIVERY_FAIL symbol with score of -40 for messages that match user part (user at domain.tld) on messages. OK, that example gives me a better idea of how to use multimaps. I'll give that a try. Thanks. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From rspamd-mlist-user-qa at eminovic.ba Sun Jul 19 18:35:08 2020 From: rspamd-mlist-user-qa at eminovic.ba (=?utf-8?Q?Nihad_=40_RSPAMD_maillist?=) Date: Sun, 19 Jul 2020 20:35:08 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> Message-ID: <2f8336ff-70ce-4f91-8a6c-cda5efe0a6e8@Spark> Did you try it out? Did it work? Just so others know if the solution was usable or not. /Nihad On 17 Jul 2020, 21.14 +0200, User questions , wrote: > > I'll > give that a try. From phils at caerllewys.net Sun Jul 19 20:35:17 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Sun, 19 Jul 2020 16:35:17 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: <2f8336ff-70ce-4f91-8a6c-cda5efe0a6e8@Spark> References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> <2f8336ff-70ce-4f91-8a6c-cda5efe0a6e8@Spark> Message-ID: On 2020-07-19 14:35, Nihad @ RSPAMD maillist via Users wrote: > Did you try it out? > Did it work? > > Just so others know if the solution was usable or not. Nihad, Still working on it, though I modified it to use email:addr and /mailer-daemon at googlemail.com/ in the map. So far the delivery failure notices are still bouncing. I'm trying it with the less restrictive email:user and /mailer-daemon/. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From a.seibel at fis-asp.de Mon Jul 20 09:20:22 2020 From: a.seibel at fis-asp.de (Seibel Andreas) Date: Mon, 20 Jul 2020 11:20:22 +0200 Subject: [Rspamd-Users] understanding scoring in multimaps In-Reply-To: References: Message-ID: <3f496597-efe2-d6ca-6de6-85d7328e4c1b@fis-asp.de> Am 16.07.2020 um 13:17 schrieb Seibel Andreas: > Hello, > > I have a problem understanding the scoring in multimaps. My local.d/multimap.conf: > > WHITELIST_FROM_AND_TO { > type = "combined"; > symbol = "EXCEPTION_FOR_BOB"; > score = -5; > rules { > to = { > map = "$LOCAL_CONFDIR/local.d/rcpt_whitelist.map.inc"; > selector = "to"; > } > from = { > map = "$LOCAL_CONFDIR/local.d/sender_whitelist.map.inc"; > selector = "from"; > } > ip = { > type = "radix"; > map = "$LOCAL_CONFDIR/local.d/ip_whitelist.map.inc"; > selector = "ip"; > } > } > # expression = "from & to & ip" > expression = "to & from" > # expression = "to & ip" > # expression = "from & ip" > } > > For the expressions "from & to & ip", "to & ip" and "from & ip" it works as expected, but not for "to & from". > In the case "to & from" symbol "EXCEPTION_FOR_BOB" is set but score is "-0.00" instead of "-5.00". > > Example for expression "from & to & ip" (works as expected): > > # rspamc --ip 10.10.10.10 --from alice at sender.dom --rcpt bob at recipient.dom < ~/testmail.eml > > Results for file: stdin (0.059 seconds) > [Metric: default] > Action: no action > Spam: false > Score: -4.10 / 15.00 > Symbol: ARC_NA (0.00) > Symbol: EXCEPTION_FOR_BOB (-5.00)[from=alice at sender.dom, to=bob at recipient.dom, ip=10.10.10.10] > Symbol: FROM_EQ_ENVFROM (0.00) > Symbol: FROM_HAS_DN (0.00) > Symbol: MID_RHS_NOT_FQDN (0.50) > Symbol: MIME_GOOD (-0.10)[multipart/alternative, text/plain] > Symbol: MIME_TRACE (0.00)[0:+, 1:+, 2:~] > Symbol: RCPT_COUNT_ONE (0.00)[1] > Symbol: RCVD_COUNT_ZERO (0.00)[0] > Symbol: R_PARTS_DIFFER (0.50)[100.0%] > Symbol: TO_DN_NONE (0.00) > Symbol: TO_MATCH_ENVRCPT_ALL (0.00) > Message-ID: test-mail at localhost > > Example for expression "to & from" (symbol is there but score is "-0.00" instead of "-5.00"): > > # rspamc --ip 10.10.10.10 --from alice at sender.dom --rcpt bob at recipient.dom < ~/testmail.eml > > Results for file: stdin (0.035 seconds) > [Metric: default] > Action: no action > Spam: false > Score: 0.90 / 15.00 > Symbol: ARC_NA (0.00) > Symbol: EXCEPTION_FOR_BOB (-0.00)[to=bob at recipient.dom, from=alice at sender.dom] > Symbol: FROM_EQ_ENVFROM (0.00) > Symbol: FROM_HAS_DN (0.00) > Symbol: MID_RHS_NOT_FQDN (0.50) > Symbol: MIME_GOOD (-0.10)[multipart/alternative, text/plain] > Symbol: MIME_TRACE (0.00)[0:+, 1:+, 2:~] > Symbol: RCPT_COUNT_ONE (0.00)[1] > Symbol: RCVD_COUNT_ZERO (0.00)[0] > Symbol: R_PARTS_DIFFER (0.50)[100.0%] > Symbol: TO_DN_NONE (0.00) > Symbol: TO_MATCH_ENVRCPT_ALL (0.00) > Message-ID: test-mail at localhost > > Does anyone have any idea what I'm doing wrong? > > Thank you. > > Andreas > > Hi, anybody got any ideas? Or does nobody use combined multimaps? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 831 bytes Desc: OpenPGP digital signature URL: From viruslaki at gmail.com Mon Jul 20 10:33:37 2020 From: viruslaki at gmail.com (Igor Smitran) Date: Mon, 20 Jul 2020 12:33:37 +0200 Subject: [Rspamd-Users] headers inside DATA part Message-ID: <777ff3e5-8c9b-b419-5e01-ac211262a3ec@gmail.com> Hi list, I am new with rspamd so please forgive me if it's stupid what i am about to ask :) I am installling new server, postfix+dovecot-lda+rspamd+... It is debian based server with rspamd installed from repo: rspamd - 2.5-156~buster amd64 [installed] and on postfix server: postfix - 3.4.10-0+deb10u1 amd64 It looks like RSPAMD adds headers inside DATA part, which is not very nice, here is mail in raw format: --- Return-Path: X-Original-To: hidden Delivered-To: hidden Received: from [*.*.*] (hidden [*.*.*]) ??? (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) ??? ?key-exchange X25519 server-signature RSA-PSS (2048 bits)) ??? (No client certificate requested) ??? (Authenticated sender: hidden) ??? by hidden (Postfix) with ESMTPSA id 65B2720060A7 ??? for ; Mon, 20 Jul 2020 10:35:09 +0200 (CEST) From: hidden Subject: test rspamd headers To: hidden Message-ID: Date: Mon, 20 Jul 2020 10:35:09 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 ?Thunderbird/68.10.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spam-Status: No, score=-0.10 test content --- As you can see, X-Spam-Status is inside DATA part. Am i doing something wrong or...? Postfix and rspamd are on two different machines and postfix is using milter to talk to rspamd: # milter RSPAMD milter_protocol = 6 milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen} milter_default_action = accept smtpd_milters = inet:rspamd_ip:11332 non_smtpd_milters = inet:rspamd_ip:11332 rspamd milter config is default except: skip_local = false; extended_spam_headers = false; skip_authenticated = false; #use = ["x-spamd-bar", "x-spam-level", "authentication-results"]; use = ["x-spam-level","x-spam-status","spam-header"]; Thank you From cubewerk at gmail.com Mon Jul 20 10:56:33 2020 From: cubewerk at gmail.com (Stefan Bauer) Date: Mon, 20 Jul 2020 12:56:33 +0200 Subject: [Rspamd-Users] headers inside DATA part In-Reply-To: <777ff3e5-8c9b-b419-5e01-ac211262a3ec@gmail.com> References: <777ff3e5-8c9b-b419-5e01-ac211262a3ec@gmail.com> Message-ID: I don't think it is. IIRC the delimiter between header and body is a newline and that's what you have shown above. Am Mo., 20. Juli 2020 um 12:33 Uhr schrieb Igor Smitran : > Hi list, > > I am new with rspamd so please forgive me if it's stupid what i am about > to ask :) > > I am installling new server, postfix+dovecot-lda+rspamd+... > > It is debian based server with rspamd installed from repo: > > rspamd - 2.5-156~buster amd64 [installed] > > and on postfix server: > > postfix - 3.4.10-0+deb10u1 amd64 > > > It looks like RSPAMD adds headers inside DATA part, which is not very > nice, here is mail in raw format: > > --- > > Return-Path: > X-Original-To: hidden > Delivered-To: hidden > Received: from [*.*.*] (hidden [*.*.*]) > (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) > key-exchange X25519 server-signature RSA-PSS (2048 bits)) > (No client certificate requested) > (Authenticated sender: hidden) > by hidden (Postfix) with ESMTPSA id 65B2720060A7 > for ; Mon, 20 Jul 2020 10:35:09 +0200 (CEST) > From: hidden > Subject: test rspamd headers > To: hidden > Message-ID: > Date: Mon, 20 Jul 2020 10:35:09 +0200 > User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 > Thunderbird/68.10.0 > MIME-Version: 1.0 > Content-Type: text/plain; charset=windows-1252; format=flowed > Content-Transfer-Encoding: 7bit > Content-Language: en-US > X-Spam-Status: No, score=-0.10 > > test content > > --- > > As you can see, X-Spam-Status is inside DATA part. Am i doing something > wrong or...? > > > Postfix and rspamd are on two different machines and postfix is using > milter to talk to rspamd: > > # milter RSPAMD > milter_protocol = 6 > milter_mail_macros = i {mail_addr} {client_addr} {client_name} > {auth_authen} > milter_default_action = accept > smtpd_milters = inet:rspamd_ip:11332 > non_smtpd_milters = inet:rspamd_ip:11332 > > > rspamd milter config is default except: > > skip_local = false; > extended_spam_headers = false; > skip_authenticated = false; > #use = ["x-spamd-bar", "x-spam-level", "authentication-results"]; > use = ["x-spam-level","x-spam-status","spam-header"]; > > > Thank you > > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users > From viruslaki at gmail.com Mon Jul 20 11:15:15 2020 From: viruslaki at gmail.com (Igor Smitran) Date: Mon, 20 Jul 2020 13:15:15 +0200 Subject: [Rspamd-Users] headers inside DATA part In-Reply-To: References: <777ff3e5-8c9b-b419-5e01-ac211262a3ec@gmail.com> Message-ID: Huh, you are right :( My bad. Sorry, On 7/20/20 12:56 PM, Stefan Bauer wrote: > I don't think it is. IIRC the delimiter between header and body is a > newline and that's what you have shown above. > > Am Mo., 20. Juli 2020 um 12:33 Uhr schrieb Igor Smitran > : >> Hi list, >> >> I am new with rspamd so please forgive me if it's stupid what i am about >> to ask :) >> >> I am installling new server, postfix+dovecot-lda+rspamd+... >> >> It is debian based server with rspamd installed from repo: >> >> rspamd - 2.5-156~buster amd64 [installed] >> >> and on postfix server: >> >> postfix - 3.4.10-0+deb10u1 amd64 >> >> >> It looks like RSPAMD adds headers inside DATA part, which is not very >> nice, here is mail in raw format: >> >> --- >> >> Return-Path: >> X-Original-To: hidden >> Delivered-To: hidden >> Received: from [*.*.*] (hidden [*.*.*]) >> (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) >> key-exchange X25519 server-signature RSA-PSS (2048 bits)) >> (No client certificate requested) >> (Authenticated sender: hidden) >> by hidden (Postfix) with ESMTPSA id 65B2720060A7 >> for ; Mon, 20 Jul 2020 10:35:09 +0200 (CEST) >> From: hidden >> Subject: test rspamd headers >> To: hidden >> Message-ID: >> Date: Mon, 20 Jul 2020 10:35:09 +0200 >> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 >> Thunderbird/68.10.0 >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=windows-1252; format=flowed >> Content-Transfer-Encoding: 7bit >> Content-Language: en-US >> X-Spam-Status: No, score=-0.10 >> >> test content >> >> --- >> >> As you can see, X-Spam-Status is inside DATA part. Am i doing something >> wrong or...? >> >> >> Postfix and rspamd are on two different machines and postfix is using >> milter to talk to rspamd: >> >> # milter RSPAMD >> milter_protocol = 6 >> milter_mail_macros = i {mail_addr} {client_addr} {client_name} >> {auth_authen} >> milter_default_action = accept >> smtpd_milters = inet:rspamd_ip:11332 >> non_smtpd_milters = inet:rspamd_ip:11332 >> >> >> rspamd milter config is default except: >> >> skip_local = false; >> extended_spam_headers = false; >> skip_authenticated = false; >> #use = ["x-spamd-bar", "x-spam-level", "authentication-results"]; >> use = ["x-spam-level","x-spam-status","spam-header"]; >> >> >> Thank you >> >> -- >> Users mailing list >> Users at lists.rspamd.com >> https://lists.rspamd.com/mailman/listinfo/users >> From phils at caerllewys.net Mon Jul 20 13:24:32 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Mon, 20 Jul 2020 09:24:32 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> <2f8336ff-70ce-4f91-8a6c-cda5efe0a6e8@Spark> Message-ID: On 2020-07-19 16:35, Phil Stracchino wrote: > On 2020-07-19 14:35, Nihad @ RSPAMD maillist via Users wrote: >> Did you try it out? >> Did it work? >> >> Just so others know if the solution was usable or not. > > > Nihad, > Still working on it, though I modified it to use email:addr and > /mailer-daemon at googlemail.com/ in the map. So far the delivery failure > notices are still bouncing. I'm trying it with the less restrictive > email:user and /mailer-daemon/. And looking at gmail and logs this morning, still getting rejections of delivery failure notices. -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From rspamd-mlist-user-qa at eminovic.ba Mon Jul 20 15:02:54 2020 From: rspamd-mlist-user-qa at eminovic.ba (=?utf-8?Q?Nihad_=40_RSPAMD_maillist?=) Date: Mon, 20 Jul 2020 17:02:54 +0200 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> <2f8336ff-70ce-4f91-8a6c-cda5efe0a6e8@Spark> Message-ID: You will have to either explain what symbols are triggering rejections or supply the list of symbols and scores for at lease one message. /Nihad On 20 Jul 2020, 15.25 +0200, Phil Stracchino , wrote: > On 2020-07-19 16:35, Phil Stracchino wrote: > > On 2020-07-19 14:35, Nihad @ RSPAMD maillist via Users wrote: > > > Did you try it out? > > > Did it work? > > > > > > Just so others know if the solution was usable or not. > > > > > > Nihad, > > Still working on it, though I modified it to use email:addr and > > /mailer-daemon at googlemail.com/ in the map. So far the delivery failure > > notices are still bouncing. I'm trying it with the less restrictive > > email:user and /mailer-daemon/. > > And looking at gmail and logs this morning, still getting rejections of > delivery failure notices. > > > -- > Phil Stracchino > Babylon Communications > phils at caerllewys.net > phil at co.ordinate.org > Landline: +1.603.293.8485 > Mobile: +1.603.998.6958 > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users From phils at caerllewys.net Mon Jul 20 15:29:08 2020 From: phils at caerllewys.net (Phil Stracchino) Date: Mon, 20 Jul 2020 11:29:08 -0400 Subject: [Rspamd-Users] Whitelist overrides - help please? In-Reply-To: References: <90635d6c-eba2-daba-bd2d-fd1345ca09c7@caerllewys.net> <27c8ed1a-0280-4e8d-baac-c042589febed@Spark> <9aaf9b9d-eeda-dec3-7ff8-c7b0b7fdf07e@caerllewys.net> <08fc340b-2803-b933-7161-e386509e839d@caerllewys.net> <761eebb4-c3f1-5ad7-a7c5-4257e8670812@caerllewys.net> <2f8336ff-70ce-4f91-8a6c-cda5efe0a6e8@Spark> Message-ID: <80cbb88b-bb5a-36a5-907d-5ea7ce89edb3@caerllewys.net> On 2020-07-20 11:02, Nihad @ RSPAMD maillist via Users wrote: > You will have to either explain what symbols are triggering rejections or supply the list of symbols and scores for at lease one message. Here's the history entry for the most recent bounce: 010001736cce3436-2a63d4da-5ebb-4f69-b6f2-a76a81f5d85a-000000 at email.amazonses.com 209.85.167.50 reject 18.39 / 15 45.2k 8.426748 7/20/2020, 11:18:53 AM Symbols ARC_REJECT, BAYES_SPAM, DKIM_TRACE, DMARC_POLICY_REJEC, HAS_REPLYTO, MIME_BASE64_TEXT, MIME_MA_MISSING_TEXT, R_DKIM_REJECT, R_SPF_ALLOW, RBL_SENDERSCORE_FAIL, RBL_VIRUSFREE_UNKNOWN_FAIL, RCVD_COUNT_THREE, REPLYTO_DN_EQ_FROM_DN, TO_DN_NONE, URI_COUNT_ODD And here's the bounced rejection in gmail: Delivered-To: alareikhs.gothemund+caf_=phils=caerllewys.net at gmail.com Received: by 2002:a2e:5cc6:0:0:0:0:0 with SMTP id q189csp1884012ljb; Mon, 20 Jul 2020 08:19:02 -0700 (PDT) X-Received: by 2002:a2e:9003:: with SMTP id h3mr10913771ljg.191.1595258342397; Mon, 20 Jul 2020 08:19:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1595258342; cv=none; d=google.com; s=arc-20160816; b=dyaNab7OVXHw/sVr8DuyBiweX1ya5rM12t79aLeVRjO94YoNI8lH23Bw7SDmEuQn23 48E8rXGKpoCRKKiKIiIZ83AOJBCcM8roIJoaw2b3n/86TQ58jMVMh6VUxmUNgIhu9rb1 PVuvZEO78cU+93Qi82flW4L7u44cySyK2hktUmw/nq1zBGd05y3d36osUfRLVci0dgp5 uE9YISm3uTgBVCaXnPrUUXrVuIREOqaWxMeCrbu+zwAGNaj8vMh1lex4HLWtMZnTceRN qSZZeuCEvzxqX8NGLzdM01Cq9bp9AzXiQzg/5huK4axWT0kCTz5DxCfsNXv2YtBooMNT T8Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:references:subject:from:date:message-id:auto-submitted :to:dkim-signature; bh=F1x9FDpxYNwboeqZhSvykf8GG/f37MaqkJaCQ7J/Sm4=; b=vGCkre7EjvwEYJZDR1X7PtrGEyHa0h/ImOFiK/nXwBYBOBC5IZRGnMj/6iOIGdoyIC obMJTkxEjKowAVOF3MDOQH/c80hepTD768Um7A6pSIJTvODDEM5NjMDduWfLvahSCtnS 5j8g+yV7PbmDTWbW+ngZxhn6k8/tKkYkfQk6QPrhG7BhPzY5nNvUVYrkCBeqktOu7rzJ 95IuYmd66rcSVm+vJwIJ4OCKhU7CCBF+3Xua4a8n2VQR3kDVHo4XY2rfDNqChAmoJlRE gwexcBe9Vh9IMCkXqNp4rac6r+YRRKc3iE4aN9ewiXYLEBEozA9wybBHwPzK2+BlUCNr WJQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=SO8xiteI; spf=pass (google.com: best guess record for domain of postmaster at mail-sor-f69.google.com designates 209.85.220.69 as permitted sender) smtp.helo=mail-sor-f69.google.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: <> Received: from mail-sor-f69.google.com (mail-sor-f69.google.com. [209.85.220.69]) by mx.google.com with SMTPS id b4sor2134287lfo.31.2020.07.20.08.19.02 for (Google Transport Security); Mon, 20 Jul 2020 08:19:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of postmaster at mail-sor-f69.google.com designates 209.85.220.69 as permitted sender) client-ip=209.85.220.69; Authentication-Results: mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=SO8xiteI; spf=pass (google.com: best guess record for domain of postmaster at mail-sor-f69.google.com designates 209.85.220.69 as permitted sender) smtp.helo=mail-sor-f69.google.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=to:auto-submitted:message-id:date:from:subject:references :in-reply-to; bh=F1x9FDpxYNwboeqZhSvykf8GG/f37MaqkJaCQ7J/Sm4=; b=SO8xiteIShRz3UnfK6PP49+qZ//IE4VLw3mVX0fxwmTJsS/3QcWeDxYsN6nDjfyKo5 XSFIllPYmSAW+XwbIskVOVc9ETTuMlWvwVowfBVMQmLFAjw3r1wxdp6BIRuQHHQoXK92 +lq8GVuQSxOAqVckxJCIDQfc4ftgaagVx4MnMGTZu7npJ7bf8ZfTK2RLXgp7OrOWZj84 wAbp1v2k3phEo0E2OlQ9f4dUDxxJTTxKMyJV8c4y2tnaQ/ubsbzo39vkafQVXpYjh0Nq U19vQFYwKGcqWD67spLlF5QGtJFEF16F8b5Kna/19WYffhuiD2sfHtA3H9CncCzoSrf/ 1PUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:auto-submitted:message-id:date:from:subject :references:in-reply-to; bh=F1x9FDpxYNwboeqZhSvykf8GG/f37MaqkJaCQ7J/Sm4=; b=M4Wqp5COZx/o8QibjRlWjrKnVIR3OdKKib5uBA0G08sLuyhnS8Go9CeFsRpMQoHr0K cI5Mhyhc39b4XXgmqAm2/5TTq7HsMyJ6/OtIJ8JqegvjsskO6pCzC0GuvinfkRYiB2k6 E5DXY+BKG7whFBaL3n8PaGCF1eFQpcM5M/jDy4aMfhinLGewAcWK2jTpn644lMrWDXKy ILXZmG+LnkEYuGx3b6LSnS2+S4ztajqzPeO5z5k707yCd0YLC9OOA+HlzDaZ/cRN51Sy Y0Ussn3kOWAbJc/xC5bfmgG5I9qXRvWe3Bm6MrwXiRYTvZizJnOAtBFt21fimtoe5A/4 JJTA== X-Gm-Message-State: AOAM532Bbmc4Kt2VnfWXFUP7eEDTEH8kGm35naUX7i6tS4x9MmCjFBPa lcKKX2E9b+jc4vH/iN2ahFghqtpiT6r0VIhQabXxRg== X-Google-Smtp-Source: ABdhPJxsCh5IPhzSeBVCmVyvz1cwDL/mjjYSq+WXaFfyYy+JKCYiEWktX89rBSz26nVwvRqGbIJWRhytB0nX/Tf7E0tSqSEME2JTY90= X-Received: by 2002:a19:228a:: with SMTP id i132mr9880629lfi.178.1595258342260; Mon, 20 Jul 2020 08:19:02 -0700 (PDT) Content-Type: multipart/report; boundary="000000000000d0b7fc05aae106d9"; report-type=delivery-status To: alareikhs.gothemund+caf_=phils=caerllewys.net at gmail.com Received: by 2002:a19:228a:: with SMTP id i132mr6341949lfi.178; Mon, 20 Jul 2020 08:19:02 -0700 (PDT) Return-Path: <> Auto-Submitted: auto-replied Message-ID: <5f15b5e6.1c69fb81.cfe8.128a.GMR at mx.google.com> Date: Mon, 20 Jul 2020 08:19:02 -0700 (PDT) From: Mail Delivery Subsystem Subject: Delivery Status Notification (Failure) References: <010001736cce3436-2a63d4da-5ebb-4f69-b6f2-a76a81f5d85a-000000 at email.amazonses.com> In-Reply-To: <010001736cce3436-2a63d4da-5ebb-4f69-b6f2-a76a81f5d85a-000000 at email.amazonses.com> X-Failed-Recipients: phils at caerllewys.net --000000000000d0b7fc05aae106d9 Content-Type: multipart/related; boundary="000000000000d0bd5405aae106df" --000000000000d0bd5405aae106df Content-Type: multipart/alternative; boundary="000000000000d0bd5905aae106e0" --000000000000d0bd5905aae106e0 Content-Type: text/plain; charset="UTF-8" ** Message not delivered ** There was a problem delivering your message to phils at caerllewys.net. See the technical details below, or try resending in a few minutes. The response from the remote server was: 554 5.7.1 Spam message rejected --000000000000d0bd5905aae106e0 Content-Type: text/html; charset="UTF-8" --000000000000d0bd5905aae106e0-- --000000000000d0bd5405aae106df Content-Type: image/png; name="icon.png" Content-Disposition: attachment; filename="icon.png" Content-Transfer-Encoding: base64 Content-ID: --000000000000d0bd5405aae106df-- --000000000000d0b7fc05aae106d9 Content-Type: message/delivery-status --000000000000d0b7fc05aae106d9 Content-Type: message/global Content-Transfer-Encoding: quoted-printable --000000000000d0b7fc05aae106d9-- -- Phil Stracchino Babylon Communications phils at caerllewys.net phil at co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 From j.echter at echter-kuechen-elektro.de Tue Jul 21 15:11:42 2020 From: j.echter at echter-kuechen-elektro.de (J. Echter) Date: Tue, 21 Jul 2020 17:11:42 +0200 Subject: [Rspamd-Users] using oletools with olefy Message-ID: <325cf0f3-af23-5354-97a9-a0d997fc1c00@echter-kuechen-elektro.de> Hi, i'd like to use the oletools with rspamd, but it doesn't work as expected. (i commented below) I have the following config files: local.d/external_services.conf oletools { ? # default olefy settings ? servers = "127.0.0.1:10632" ? # needs to be set explicitly for Rspamd < 1.9.5 ? scan_mime_parts = true; ? # mime-part regex matching in content-type or filename ? mime_parts_filter_regex { ??? #UNKNOWN = "application\/octet-stream"; ??? DOC2 = "application\/msword"; ??? DOC3 = "application\/vnd\.ms-word.*"; ??? XLS = "application\/vnd\.ms-excel.*"; ??? PPT = "application\/vnd\.ms-powerpoint.*"; ??? GENERIC = "application\/vnd\.openxmlformats-officedocument.*"; ? } ? # mime-part filename extension matching (no regex) ? mime_parts_filter_ext { ??? doc = "doc"; ??? dot = "dot"; ??? docx = "docx"; ??? dotx = "dotx"; ??? docm = "docm"; ??? dotm = "dotm"; ??? xls = "xls"; ??? xlt = "xlt"; ??? xla = "xla"; ??? xlsx = "xlsx"; ??? xltx = "xltx"; ??? xlsm = "xlsm"; ??? xltm = "xltm"; ??? xlam = "xlam"; ??? xlsb = "xlsb"; ??? ppt = "ppt"; ??? pot = "pot"; ??? pps = "pps"; ??? ppa = "ppa"; ??? pptx = "pptx"; ??? potx = "potx"; ??? ppsx = "ppsx"; ??? ppam = "ppam"; ??? pptm = "pptm"; ??? potm = "potm"; ??? ppsm = "ppsm"; ? } ? patterns { ??? # catch Macro, AutoExec, Suspicious and Hex Strings ??? BAD_MACRO_MYFLAGS = '^MAS.H...$'; ??? BAD_MACRO_SHELL?? = '^Shell$'; ? } } local.d/external_services_group.conf "OLETOOLS" { ??? weight = 1.0; ??? description = "OLETOOLS found a Macro"; ??? one_shot = true; ? } Also i set olefy (i set debug logging in the conf) up and it is running: Jul 21 17:02:54 mail systemd[1]: Started olefy Socket Service. Jul 21 17:02:54 mail python3[22283]: olefy DEBUG olefy listen address string: 127.0.0.1, ::1 (type ) Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy listen address: ['127.0.0.1', '::1'] (type: ) Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy listen port: 10632 Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy tmp dir: /tmp Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy python path: /usr/bin/python3 Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy olvba path: /usr/bin/olevba-3 Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy log level: 10 Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy min file length: 500 Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy delete tmp file: 1 Jul 21 17:02:54 mail python3[22283]: olefy INFO olefy delete tmp file when failed: 1 Jul 21 17:02:54 mail python3[22283]: olefy DEBUG __init__ Using selector: EpollSelector Jul 21 17:02:54 mail python3[22283]: olefy INFO serving on ('127.0.0.1', 10632) Jul 21 17:02:54 mail python3[22283]: olefy INFO serving on ('::1', 10632, 0, 0) I have seen one connection made from rspamd by sending a .doc testmail: Jul 21 16:53:35 mail python3[21894]: olefy DEBUG connection_made ('127.0.0.1', 48336) new connection was made Jul 21 16:53:35 mail python3[21894]: olefy DEBUG data_received ('127.0.0.1', 48336) data received from new connection Jul 21 16:53:35 mail python3[21894]: olefy DEBUG protocol_split olefy_headers: {'olefy': 'OLEFY/1.0', 'Method': 'oletools', 'Rspamd-ID': '068495c07a7c5942887'} Jul 21 16:53:35 mail python3[21894]: olefy DEBUG eof_received <068495> /tmp/1595343215.9703288.48336 choosen as tmp filename Jul 21 16:53:35 mail python3[21894]: olefy INFO eof_received <068495> 30208 bytes (stream size) Jul 21 16:53:35 mail python3[21894]: olefy INFO oletools <068495> application/msword (libmagic output) Jul 21 16:53:36 mail python3[21894]: olefy DEBUG oletools <068495> /tmp/1595343215.9703288.48336 deleting tmp file Jul 21 16:53:36 mail python3[21894]: olefy DEBUG oletools <068495> response: [? {??? "script_name": "olevba",??? "version": "0.54.2",??? "url": "http://decalage.info/python/oletools",??? "type": "MetaInformation"? },? {??? "container": null,??? "file": "/tmp/1595343215.9703288.48336",??? "json_conversion_successful": true,??? "analysis": null,??? "code_deobfuscated": null,??? "do_deobfuscate": false,??? "type": "OLE",??? "macros": []? },? {??? "type": "MetaInformation",??? "return_code": 0,??? "n_processed": 1? }] Jul 21 16:53:36 mail python3[21894]: olefy INFO eof_received <068495> ('127.0.0.1', 48336) response send: b'[? {??? "script_name": "olevba",??? "version": "0.54.2",??? "url": "http://decalage.info/python/oletools",??? "type": "MetaInformation"? },? {??? "container": null,??? "file": "/tmp/1595343215.9703288.48336",??? "json_conversion_successful": true,??? "analysis": null,??? "code_deobfuscated": null,??? "do_deobfuscate": false,??? "type": "OLE",??? "macros": []? },? {??? "type": "MetaInformation",??? "return_code": 0,??? "n_processed": 1? }]\t\n\n\t' But i havent seen any other doc files scanned after the above one, i sent quite a few mails after that. Also i don't see "oletools" in the X-Spamd-Results in the mail source. I'm on centos 7 and rspamd is from the rspamd repo (version 2.5), also i cloned the olefy github repo and set everything up as described there. I installed python36-oletools with yum and did pip3 install python-magic, the python-magic i could install with yum didn't seem to work. Anything i have overlooked? Thanks for helping me :) Juergen From fluffy-bunny at hush.com Thu Jul 23 14:25:44 2020 From: fluffy-bunny at hush.com (fluffy-bunny at hush.com) Date: Thu, 23 Jul 2020 16:25:44 +0200 Subject: [Rspamd-Users] Problems with redis replica on rspamd Message-ID: <20200723142544.BE17AA032C6@smtp.hushmail.com> Hi there! 'err I've set up two new instances of rspamd with redis and it seems to work fine. But if I upload an Spam-Mail via the Web-GUI I get this error under 'History' ---> 'Errors' : I can see this also in /var/log/rspamd/rspamd.log on Node02: ```ish.com, next check at Thu, 23 Jul 2020 14:22:35 GMT (timer based) 2020-07-23 16:17:49 #6474(controller) ; lua; bayes_expiry.lua:405: cannot perform expiry step: ERR Error running script (call to f_acd6aff7fcbc3b001f95b7e5a710653d7b51aae3): @user_script:36: @user_script: 36: -READONLY You can't write against a read only replica. 2020-07-23 16:19:08 #6474(controller) ; lua; bayes_expiry.lua:405: cannot perform expiry step: ERR Error running script (call to f_acd6aff7fcbc3b001f95b7e5a710653d7b51aae3): @user_script:36: @user_script: 36: -READONLY You can't write against a read only replica. 2020-07-23 16:20:15 #6474(controller) ; lua; bayes_expiry.lua:405: cannot perform expiry step: ERR Error running script (call to f_acd6aff7fcbc3b001f95b7e5a710653d7b51aae3): @user_script:36: @user_script: 36: -READONLY You can't write against a read only replica. 2020-07-23 16:21:58 #6474(controller) ; lua; bayes_expiry.lua:405: cannot perform expiry step: ERR Error running script (call to f_acd6aff7fcbc3b001f95b7e5a710653d7b51aae3): @user_script:36: @user_script: 36: -READONLY You can't write against a read only replica.``` ...and on Node01 I can see this in /var/log/rspamd/rspamd.log: ```2020-07-23 16:21:20 #6667(controller) ; neural; neural.lua:1150: try_train_ann: process element MONTHLY:default 2020-07-23 16:21:20 #6667(controller) ; neural; neural.lua:1150: try_load_ann: process element WEEKLY:default 2020-07-23 16:21:20 #6667(controller) ; neural; neural.lua:1150: try_load_ann: process element MONTHLY:default 2020-07-23 16:21:20 #6667(controller) ; neural; neural.lua:1150: try_train_ann: process element WEEKLY:default 2020-07-23 16:21:20 #6667(controller) ; neural; neural.lua:1150: try_load_ann: process element SHORT:default 2020-07-23 16:21:20 #6667(controller) ; neural; neural.lua:1150: try_train_ann: process element SHORT:default 2020-07-23 16:21:20 #6668(controller) ; neural; neural.lua:1150: try_load_ann: process element MONTHLY:default 2020-07-23 16:21:20 #6668(controller) ; neural; neural.lua:1150: try_load_ann: process element SHORT:default 2020-07-23 16:21:20 #6668(controller) ; neural; neural.lua:1150: try_load_ann: process element WEEKLY:default 2020-07-23 16:22:20 #6666(rspamd_proxy) ; neural; neural.lua:1150: try_load_ann: process element WEEKLY:default 2020-07-23 16:22:20 #6666(rspamd_proxy) ; neural; neural.lua:1150: try_load_ann: process element SHORT:default 2020-07-23 16:22:20 #6666(rspamd_proxy) ; neural; neural.lua:1150: try_load_ann: process element MONTHLY:default 2020-07-23 16:22:20 #6667(controller) ; neural; neural.lua:1150: try_train_ann: process element MONTHLY:default 2020-07-23 16:22:20 #6667(controller) ; neural; neural.lua:1150: try_train_ann: process element WEEKLY:default 2020-07-23 16:22:20 #6667(controller) ; neural; neural.lua:1150: try_load_ann: process element MONTHLY:default 2020-07-23 16:22:20 #6667(controller) ; neural; neural.lua:1150: try_load_ann: process element SHORT:default 2020-07-23 16:22:20 #6667(controller) ; neural; neural.lua:1150: try_load_ann: process element WEEKLY:default 2020-07-23 16:22:20 #6667(controller) ; neural; neural.lua:1150: try_train_ann: process element SHORT:default 2020-07-23 16:22:20 #6668(controller) ; neural; neural.lua:1150: try_load_ann: process element SHORT:default 2020-07-23 16:22:20 #6668(controller) ; neural; neural.lua:1150: try_load_ann: process element WEEKLY:default 2020-07-23 16:22:20 #6668(controller) ; neural; neural.lua:1150: try_load_ann: process element MONTHLY:default``` Does anyone have an idea why I get the error 'You can't write against a read only replica.`? I've no I idea what I can do regarding this? From martin at black-sheep-research.com Thu Jul 23 18:14:28 2020 From: martin at black-sheep-research.com (Martin Brampton) Date: Thu, 23 Jul 2020 19:14:28 +0100 Subject: [Rspamd-Users] How to configure rspamd to block viruses but not reject In-Reply-To: <20200723142544.BE17AA032C6@smtp.hushmail.com> References: <20200723142544.BE17AA032C6@smtp.hushmail.com> Message-ID: Hi I might be confused, but seem to have a problem with clamav/rspamd antivirus soft bouncing bad mail. The problem is that, on a soft bounce, the mail relay sends the failed email back to the sender. But the sender's mail server is liable to also soft bounce the failure message, because it includes the virus. Is there a way to log the issue and dump the message? I haven't been able to find what alternatives there are to action = "reject". Best regards, Martin