From t.hendricks at interpool.de  Thu Feb  1 00:36:56 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Thu, 1 Feb 2024 01:36:56 +0100
Subject: [Rspamd-Users] Columns of Web UI
Message-ID: <A55557BB-5169-44FA-86EC-9D1C4250796E@interpool.de>

Hi there,

I needed to move our mailserver to another machine (and also gentoo => debian).
Now I?m missing the handy infos like subject, sender and/or recipient in the history tab of the Web UI.

I?m on bookworm with version 3.4.

Any chance to change the appearance?

Thanks,

Tino

From moiseev at mezonplus.ru  Thu Feb  1 06:14:28 2024
From: moiseev at mezonplus.ru (Alexander Moisseev)
Date: Thu, 1 Feb 2024 09:14:28 +0300
Subject: [Rspamd-Users] Columns of Web UI
In-Reply-To: <A55557BB-5169-44FA-86EC-9D1C4250796E@interpool.de>
References: <A55557BB-5169-44FA-86EC-9D1C4250796E@interpool.de>
Message-ID: <7b759954-e7c6-6931-df7a-1832326673bc@mezonplus.ru>

On 01.02.2024 3:36, Tino Hendricks wrote:
> Hi there,
> 
> I needed to move our mailserver to another machine (and also gentoo => debian).
> Now I?m missing the handy infos like subject, sender and/or recipient in the history tab of the Web UI.
> 
> I?m on bookworm with version 3.4.
> 
> Any chance to change the appearance?
> 

It's likely that you have disabled the history_redis module.

From konstantin.kletschke at inside-m2m.de  Thu Feb  1 09:37:26 2024
From: konstantin.kletschke at inside-m2m.de (Konstantin Kletschke)
Date: Thu, 1 Feb 2024 10:37:26 +0100
Subject: [Rspamd-Users] Skip spam check for authenticated (SASL) users
 howto with postfix?
In-Reply-To: <2930cfda-9a8d-4503-a8df-4333a6b925ef@huarp.harvard.edu>
References: <ZbpQMi5FFSrZjz-f@hephaistos.inside-m2m.de>
 <b0e5f24b-2cb6-4fdf-902d-f10a5dacd8e6@huarp.harvard.edu>
 <ZbphvtOz_jhrKf8z@hephaistos.inside-m2m.de>
 <d19142e5-e669-444e-9efe-96dd18af083b@huarp.harvard.edu>
 <Zbpp5bxQgKFU58At@hephaistos.inside-m2m.de>
 <2930cfda-9a8d-4503-a8df-4333a6b925ef@huarp.harvard.edu>
Message-ID: <ZbtmVkYRbh2eqf9b@hephaistos.inside-m2m.de>

On Wed, Jan 31, 2024 at 11:01:10AM -0500, Allen, Norton T. wrote:

> Well that at least means those apply rules might work if the message is
> identified correctly, but from what you've shown me, I don't see why it
> wasn't identified as authenticated.

Sorry, I was too much in a hurry yesterday.
I investigated this more carefully until now:
Every mail delivered into the system via SASL AUTH now gets flagged like
this in the rspamd log:

(normal) <97fd5d>; lua; settings.lua:390: <XXX at inside-m2m.de> apply static settings authenticated (id = 1937017268); authenticated matched; priority high


(normal) <97fd5d>; lua; settings.lua:390: <XXX at inside-m2m.de> apply static settings authenticated (id = 1937017268); authenticated matched; priority high
(normal) <97fd5d>; task; rspamd_stat_classifiers_process: skip statistics as SPAM class is missing
(normal) <97fd5d>; task; rspamd_task_write_log: id: <XXX at inside-m2m.de>, qid: <5535640002>, ip: 80.228.41.210, user: XXX at inside-m2m.de, from: <XXX at inside-m2m.de>, (default: F (no action): [0.00/15.00] [TAGGED_RCPT(0.00){}]), len: 1725, time: 3.250ms, dns req: 0, digest: <c0c6b51bcd175fd8196804f06898fcbc>, rcpts: <YYY at AAA.com,ZZZ at BBB.com>, mime_rcpts: <YYY at AAA.com,>, settings_id: authenticated

So the setting kicks in and its consistend all other mails get this
flagging not. All fine.
What I wonder is, is that it? Is this skipping spam checking? Because
there still is a "(default: F (no action)".

And if I do the GTUBE test from intern via SALS AUTH this happens still:

(normal) <141559>; task; rspamd_worker_body_handler: accepted connection from ::1 port 40628, task ptr: 00007FB04C5B99E0
(normal) <141559>; task; rspamd_message_parse: loaded message; id: <XXX at inside-m2m.de>; queue-id: <9B10F401F5>; size: 2359; checksum: <2251d4a84b69cb97e681af7c551eb3b8>
(normal) <141559>; task; rspamd_check_gtube: gtube reject pattern has been found in part of length 390
(normal) <141559>; task; rspamd_add_passthrough_result: <XXX at inside-m2m.de>: set pre-result to 'reject' (15.00): 'Gtube pattern' from GTUBE(3)
(normal) <141559>; task; rspamd_check_gtube: gtube reject pattern has been found in part of length 370
(normal) <141559>; task; rspamd_add_passthrough_result: <XXX at inside-m2m.de>: set pre-result to 'reject' (15.00): 'Gtube pattern' from GTUBE(3)
(normal) <141559>; task; rspamd_task_write_log: id: <XXX at inside-m2m.de>, qid: <9B10F401F5>, ip: 90.187.159.109, user: AAA at inside-m2m.de, from: <AAA at inside-m2m.de>, (default: S (reject): [15.00/15.00] [GTUBE(0.00){}]), len: 2359, time: 1.214ms, dns req: 0, digest: <2251d4a84b69cb97e681af7c551eb3b8>, rcpts: <BBB at inside-m2m.de>, mime_rcpts: <BBB at inside-m2m.de>, forced: reject "Gtube pattern"; score=15.00 (set by GTUBE)
(normal) <141559>; task; rspamd_protocol_http_reply: regexp statistics: 0 pcre regexps scanned, 0 regexps matched, 175 regexps total, 0 regexps cached, 0B scanned using pcre, 0B scanned total


Is the GTUBE test not skipped globally never or is my setup still not
skippin SASL AUTH mails? How do I test this further?


Regards
Konstantin


-- 
INSIDE M2M GmbH
Konstantin Kletschke
Berenbosteler Stra?e 76 B
30823 Garbsen

Telefon: +49 (0) 5137 90950136
Mobil: +49 (0) 151 15256238
Fax: +49 (0) 5137 9095010

konstantin.kletschke at inside-m2m.de
http://www.inside-m2m.de 

Gesch?ftsf?hrung: Michael Emmert, Derek Uhlig
HRB: 111204, AG Hannover


From konstantin.kletschke at inside-m2m.de  Thu Feb  1 12:41:51 2024
From: konstantin.kletschke at inside-m2m.de (Konstantin Kletschke)
Date: Thu, 1 Feb 2024 13:41:51 +0100
Subject: [Rspamd-Users] Skip spam check for authenticated (SASL) users
 howto with postfix?
In-Reply-To: <ZbtmVkYRbh2eqf9b@hephaistos.inside-m2m.de>
References: <ZbpQMi5FFSrZjz-f@hephaistos.inside-m2m.de>
 <b0e5f24b-2cb6-4fdf-902d-f10a5dacd8e6@huarp.harvard.edu>
 <ZbphvtOz_jhrKf8z@hephaistos.inside-m2m.de>
 <d19142e5-e669-444e-9efe-96dd18af083b@huarp.harvard.edu>
 <Zbpp5bxQgKFU58At@hephaistos.inside-m2m.de>
 <2930cfda-9a8d-4503-a8df-4333a6b925ef@huarp.harvard.edu>
 <ZbtmVkYRbh2eqf9b@hephaistos.inside-m2m.de>
Message-ID: <ZbuRj3YTWKoc2PQ8@hephaistos.inside-m2m.de>

I found some fellow with the exact same issue in a github discussion:
https://github.com/rspamd/rspamd/discussions/4199

-- 
INSIDE M2M GmbH
Konstantin Kletschke
Berenbosteler Stra?e 76 B
30823 Garbsen

Telefon: +49 (0) 5137 90950136
Mobil: +49 (0) 151 15256238
Fax: +49 (0) 5137 9095010

konstantin.kletschke at inside-m2m.de
http://www.inside-m2m.de 

Gesch?ftsf?hrung: Michael Emmert, Derek Uhlig
HRB: 111204, AG Hannover


From konstantin.kletschke at inside-m2m.de  Thu Feb  1 13:33:11 2024
From: konstantin.kletschke at inside-m2m.de (Konstantin Kletschke)
Date: Thu, 1 Feb 2024 14:33:11 +0100
Subject: [Rspamd-Users] Skip spam check for authenticated (SASL) users
 howto with postfix?
In-Reply-To: <ZbuRj3YTWKoc2PQ8@hephaistos.inside-m2m.de>
References: <ZbpQMi5FFSrZjz-f@hephaistos.inside-m2m.de>
 <b0e5f24b-2cb6-4fdf-902d-f10a5dacd8e6@huarp.harvard.edu>
 <ZbphvtOz_jhrKf8z@hephaistos.inside-m2m.de>
 <d19142e5-e669-444e-9efe-96dd18af083b@huarp.harvard.edu>
 <Zbpp5bxQgKFU58At@hephaistos.inside-m2m.de>
 <2930cfda-9a8d-4503-a8df-4333a6b925ef@huarp.harvard.edu>
 <ZbtmVkYRbh2eqf9b@hephaistos.inside-m2m.de>
 <ZbuRj3YTWKoc2PQ8@hephaistos.inside-m2m.de>
Message-ID: <Zbudl_ct9zJjYQbL@hephaistos.inside-m2m.de>

Turns out that gtube test is done always unconditionally, at least when it is not
configured to be not done globally unconditionally. So this behaviour is expected.
So my setup could work and I need to figure out on how to write me some spam
mails otherwise to test this :)

From mwl at mwl.io  Fri Feb  2 19:01:45 2024
From: mwl at mwl.io (Michael W. Lucas)
Date: Fri, 2 Feb 2024 14:01:45 -0500
Subject: [Rspamd-Users] list symbols at command line?
In-Reply-To: <541f6ebaa4f6587a3153cd5da3faf211baa2e3b5.camel@judo.za.org>
References: <ZblE8AIhAQnAsVuD@mail.ratoperatedvehicle.com>
 <541f6ebaa4f6587a3153cd5da3faf211baa2e3b5.camel@judo.za.org>
Message-ID: <Zb08GVKVMkjIhbRe@mail.ratoperatedvehicle.com>

On Wed, Jan 31, 2024 at 01:00:11PM +0200, Andrew Lewis via Users wrote:
> Hi Michael,
> 
> On Tue, 2024-01-30 at 13:50 -0500, Michael W. Lucas wrote:
> > Is there a way to search for specific symbols and their meaning at
> > the
> > command line? Or to get a list of all symbols and their description?
> 
> You can find that in WebUI, or with `curl
> http://127.0.0.1:11334/symbols`

Thanks.

For the archives:

If you want a convenient grep-able list of symbol names, weights, and
descriptions from the download above, here's the jq command.

$ jq -c '.[].rules[] | [.symbol,.weight,.description]' symbols.json  > symbols.txt

==ml


-- 
Michael W. Lucas        https://mwl.io/
author of: Absolute OpenBSD, SSH Mastery, git commit murder,
 Absolute FreeBSD, Butterfly Stomp Waltz, TLS Mastery, etc...
### New books: DNSSEC Mastery, Letters to ed(1), $ git sync murder ###

From usenet at schani.com  Sat Feb  3 15:38:02 2024
From: usenet at schani.com (christian)
Date: Sat, 3 Feb 2024 16:38:02 +0100
Subject: [Rspamd-Users] Redis is constantly busy and need 12GB Ram
Message-ID: <216584de-52b8-4664-9348-3ec467802d74@schani.com>

Hello,
Is it normal that RspanD establishes a hundred client connections to 
Redis at the same time.
I only have 10 email accounts and 1000 emails a day. But Redis is 
constantly busy. Redis is only used by Rspamd.
I limited Redis' maxmemory to 1G. Previously it was 12GB RAM with redis. 
How do I get this under control?
thanks for your help
Christian

From caponecicero at gmail.com  Sat Feb  3 18:47:54 2024
From: caponecicero at gmail.com (Steve Witten)
Date: Sat, 3 Feb 2024 10:47:54 -0800
Subject: [Rspamd-Users] I'm confused...
Message-ID: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>

Hello.

I have two systems:

   - *mail.example.com <http://mail.example.com>* (where postfix runs)
   - *rspamd.example.com <http://rspamd.example.com>* (where rspamd/redis
   runs)

Each of these are FreeBSD 14.0p4.  Each has three IP addresses -- 2 public
addresses (IPv4, IPv6) and a private (unroutable, IPv4) one.

FreeBSD generates daily/weekly/monthly reports about the results of routine
system checks and mails them to root (so-called *periodic* reports).

The periodic mail from *mail.example.com <http://mail.example.com>* is *not*
DKIM-signed and rspamd generates the following header for it:

X-Spamd-Result: default: False [7.99 / 15.00]; BAYES_HAM(-3.00)[99.99%];
> NEURAL_SPAM_SHORT(1.99)[0.996]; DMARC_POLICY_QUARANTINE(1.50)[example.com
> : No valid SPF, No valid DKIM,quarantine]; RCVD_NO_TLS_LAST(0.10)[];
> MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; R_SPF_NA(0.00)[no SPF
> record]; RCVD_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; ARC_NA(0.00)[];
> FROM_NO_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[];
> FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:8000::/38,
> country:US]; TO_DN_NONE(0.00)[]; R_DKIM_NA(0.00)[];
> TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_SIGNED(0.00)[example.com:s=mailkey:i=1]
> X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC
> X-Spam-Status: Yes, score=7.99


The periodic mail from *rspamd**.example.com <http://example.com>*
*is* DKIM-signed
and rspamd generates the following header for it:


X-Spamd-Result: default: False [-1.91 / 15.00]; BAYES_HAM(-3.00)[99.99%];
> NEURAL_SPAM_SHORT(1.09)[0.547]; MIME_GOOD(-0.10)[text/plain];
> RCVD_NO_TLS_LAST(0.10)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_ONE(0.00)[1];
> MIME_TRACE(0.00)[0:+]; FROM_NO_DN(0.00)[]; ARC_NA(0.00)[];
> MID_RHS_MATCH_FROMTLD(0.00)[]; ARC_SIGNED(0.00)[example.com:s=mailkey:i=1];
> FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[example.com:s=mailkey];
> TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MISSING_XM_UA(0.00)[]
> X-Spam-Status: No, score=-1.91
> Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com;
> s=mailkey; t=1706981766;
> h=from:from:reply-to:subject:subject:date:date:message-id:message-id:  to:to:cc;
> bh=j9fcdOWE0H02egnjagQqxaytDyAclmMs6pxRnWPQfpM=;
> b=KX178rXRVp3WamOHnf0xet9YkTG4/ypnrGZ051Wjn3+mA5UIN4yM5YsfOIpTRHm9I10pM8
> a0zoYcA89/ruFkQjSaM3JP6unS23VeLXe4a3bwWtmwjBWXjw1pG+jwfs1FGW1WmebWLUx6
> 83+0gmCxWKg+PBTRVnaBby6WSZHRjEc=


Both messages are scanned by the same rspamd instance using a single
configuration.  Here's my *dkim_signing.conf*:

domain {
>     example.com {
>         selector = "mailkey";
>         path = "/srv/rspamd/var/db/dkim/example.com.mailkey.key";
>     }
> }
>
> apply {
>     flags = ["skip_process"]; # Disable expensive MIME processing
> }
>
> # If false, messages with empty envelope from are not signed
> allow_envfrom_empty = true;
>
> # If true, envelope/header domain mismatch is ignored
> allow_hdrfrom_mismatch = false;
>
> # If true, multiple from headers are allowed (but only first is used)
> allow_hdrfrom_multiple = true;
>
> # If true, username does not need to contain matching domain
> allow_username_mismatch = true;
>
> # Default path to key, can include '$domain' and '$selector' variables
> #path = "/var/lib/rspamd/dkim/$domain.$selector.key";
>
> # Default selector to use
> #selector = "dkim";
>
> # If false, messages from authenticated users are not selected for signing
> sign_authenticated = true;
>
> # If false, messages from local networks are not selected for signing
> sign_local = true;
>
> # Map file of IP addresses/subnets to consider for signing
> # sign_networks = "/some/file"; # or url
>
> # Symbol to add when message is signed
> symbol = "DKIM_SIGNED";
>
> # Whether to fallback to global config
> try_fallback = false;
>
> # Domain to use for DKIM signing: can be "header" (MIME From), "envelope"
> (SMTP From) or "auth" (SMTP username)
> use_domain = "header";
>
> # Domain to use for DKIM signing when sender is in sign_networks
> ("header"/"envelope"/"auth")
> #use_domain_sign_networks = "header";
>
> # Domain to use for DKIM signing when sender is a local IP
> ("header"/"envelope"/"auth")
> #use_domain_sign_local = "header";
>
> # Whether to normalise domains to eSLD
> use_esld = true;
>
> # Whether to get keys from Redis
> use_redis = false;
>
> # Hash for DKIM keys in Redis
> #key_prefix = "DKIM_KEYS";
>
> # map of domains -> names of selectors (since rspamd 1.5.3)
> #selector_map = "/etc/rspamd/dkim_selectors.map";
>
> # map of domains -> paths to keys (since rspamd 1.5.3)
> #path_map = "/etc/rspamd/dkim_paths.map";
>
> # If `true` get pubkey from DNS record and check if it matches private key
> check_pubkey = true;
>
> # Set to `false` if you want to skip signing if public and private keys
> mismatch
> allow_pubkey_mismatch = true;
>

Obviously, there are SPF, DMARC, & DKIM records for example.com...otherwise
rspamd couldn't find them to sign messages from *rspamd.example.com
<http://rspamd.example.com>*.  The DKIM keys are re-generated automatically
once a month.

Why is one message signed and the other not?  Is this a postfix
configuration issue or an rspamd configuration issue?  At this point I've
run out of ideas about where to look.

I'd appreciate some pointers please.  More information cheerfully
supplied.  Thanks in advance for your kind responses.

Regards,

Steve Witten

From caponecicero at gmail.com  Sat Feb  3 18:50:41 2024
From: caponecicero at gmail.com (Steve Witten)
Date: Sat, 3 Feb 2024 10:50:41 -0800
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
Message-ID: <CALGB+_xzzUfg+L87h-btBHGdNURgG2YG7_985-JriY7W5OHwxw@mail.gmail.com>

The rspamd version is

Rspamd daemon version 3.8.1


Steve Witten

From rspamd at jubileegroup.co.uk  Sat Feb  3 22:51:28 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Sat, 3 Feb 2024 22:51:28 +0000 (GMT)
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
Message-ID: <3586e42-ccd6-2641-fe92-b235468e46d@jubileegroup.co.uk>

Hi there,

On Sat, 3 Feb 2024, Steve Witten wrote:

> I have two systems:
>
>   - *mail.example.com ...
>   - *rspamd.example.com ...
> ...
> ...
> Obviously, there are SPF, DMARC, & DKIM records for example.com...

Please give the real names.

On a point of order, the SPF record for example.com has nothing to do
with the (entirely separate) SPF records for mail.example.com and
rspamd.example.com, and it is not permissible (for, er, example if SPF
records for mail.example.com and rspamd.example.com cannot be found)
to use the SPF record for example.com instead.

-- 

73,
Ged.


From caponecicero at gmail.com  Sun Feb  4 00:27:39 2024
From: caponecicero at gmail.com (Steve Witten)
Date: Sat, 3 Feb 2024 16:27:39 -0800
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <3586e42-ccd6-2641-fe92-b235468e46d@jubileegroup.co.uk>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
 <3586e42-ccd6-2641-fe92-b235468e46d@jubileegroup.co.uk>
Message-ID: <CALGB+_xQhb=Wkf7SPz9W8AO7jWSvzcWMLXPoJ8OM4hx7m5attw@mail.gmail.com>

mail.niteflyte.net
rspamd.niteflyte.net

There's no DMARC record for either of these but there IS a DMARC record for
niteflyte.net that should be used.  Only mail.niteflyte.net has an MX
record.

The mail is sent by 'mail' on each system:

mail -E -s 'the subject' root

root is aliased to 'postmaster at niteflyte.net' on each system.  '
postmaster at niteflyte.net' is aliased to me on mail.niteflyte.net.  The MTA
on rspamd.niteflyte.net is 'dma' (Dragonfly mail agent).  The MTA on
mail.niteflyte.net is postfix.

However, I think this is the answer:

On a point of order, the SPF record for example.com has nothing to do
> with the (entirely separate) SPF records for mail.example.com and
> rspamd.example.com, and it is not permissible (for, er, example if SPF
> records for mail.example.com and rspamd.example.com cannot be found)
> to use the SPF record for example.com instead.


Maybe the better question to ask is:  How can I prevent rspamd from
scanning this mail?  Since it's internal, status-report kind of stuff, it's
not really worthwhile to do this.

-- SW

On Sat, Feb 3, 2024 at 2:53?PM G.W. Haywood <rspamd at jubileegroup.co.uk>
wrote:

> Hi there,
>

<snip />

From list+rspamd at gcore.biz  Sun Feb  4 02:14:24 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Sun, 4 Feb 2024 03:14:24 +0100
Subject: [Rspamd-Users] Redis is constantly busy and need 12GB Ram
In-Reply-To: <216584de-52b8-4664-9348-3ec467802d74@schani.com>
References: <216584de-52b8-4664-9348-3ec467802d74@schani.com>
Message-ID: <2496146F-7B39-431E-8B70-31E8949B321B@gcore.biz>

> Is it normal that RspanD establishes a hundred client connections to Redis at the same time.

How do you measure that? Are these established sockets or sockets in time_wait state?
How many mails do you receive in parallel?
Are there separate redis instances for bayes, replies, reputation, ...?

> I only have 10 email accounts and 1000 emails a day. But Redis is constantly busy. Redis is only used by Rspamd.

How do you define busy? One redis instance that uses 100% cpu all day?

> I limited Redis' maxmemory to 1G. Previously it was 12GB RAM with redis. How do I get this under control?

What kind of memory is that, resident (rss) or virtual?

Some people like to configure a separate redis instance for the bayes classifier and let redis evict old keys:
https://rspamd.com/doc/modules/bayes_expiry.html#limiting-memory-usage-to-a-fixed-amount

Best regards,
Gerald

From list+rspamd at gcore.biz  Sun Feb  4 02:51:46 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Sun, 4 Feb 2024 03:51:46 +0100
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
Message-ID: <F7BB28C7-E27E-44E9-AB73-7B72330AD049@gcore.biz>

>   - *mail.example.com <http://mail.example.com>* (where postfix runs)
>   - *rspamd.example.com <http://rspamd.example.com>* (where rspamd/redis
>   runs)
> 
> Each of these are FreeBSD 14.0p4.  Each has three IP addresses -- 2 public
> addresses (IPv4, IPv6) and a private (unroutable, IPv4) one.
> 
> FreeBSD generates daily/weekly/monthly reports about the results of routine
> system checks and mails them to root (so-called *periodic* reports).

As you wrote in another mail you're sending those reports using (/usr/bin/mail),
so mails are picked up locally by postix and are not injected via smtp.

Mails received via smtp/non-smtp are handled differently inside postfix, see
https://www.postfix.org/postconf.5.html#non_smtpd_milters
https://www.postfix.org/MILTER_README.html#non-smtp-milters

Is non_smtpd_milters in main.cf set accordingly?

Rspamd signs mails that are authenticated, e.g. via milter/sasl or by ip.
You say that mails from rspamd.example.com are signed, so they are probably
of local origin from rspamd's view (sign_local = true).

Mail.example.com's log contains: ipnet:2001:19f0:8000::/38 - this seems
to be delivered via ipv6. Why is that? Shouldn't it be handed over to
rspamd via your private ipv4? In that case it might qualify for sign_local.

Otherwise there is sign_networks:
# Map file of IP addresses/subnets to consider for signing
# sign_networks = "/some/file"; # or url

Best regards,
Gerald


>> # If false, messages from authenticated users are not selected for signing
>> sign_authenticated = true;
>> 
>> # If false, messages from local networks are not selected for signing
>> sign_local = true;


From caponecicero at gmail.com  Sun Feb  4 03:56:12 2024
From: caponecicero at gmail.com (Steve Witten)
Date: Sat, 3 Feb 2024 19:56:12 -0800
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <F7BB28C7-E27E-44E9-AB73-7B72330AD049@gcore.biz>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
 <F7BB28C7-E27E-44E9-AB73-7B72330AD049@gcore.biz>
Message-ID: <CALGB+_x-g=iQFmfeHEt=5yVJUyhsiVe2SyNF5QUGGiji5rFOrw@mail.gmail.com>

G Heywood & Gerald Galster --

Thanks for your help.  It was G Heywood's tip on the SPF records that
solved the problem.  The single-SPF setup was an artifact of when all this
stuff lived on the same computer.

In all of that, I discovered that the cron job that updated the DKIM public
key TXT record in DNS wasn't working correctly either so messages were not
being signed correctly.  I fixed that too.

Thanks for your help!

Steve Witten

On Sat, Feb 3, 2024 at 6:53?PM Gerald Galster <list+rspamd at gcore.biz> wrote:

> >   - *mail.example.com <http://mail.example.com>* (where postfix runs)
> >   - *rspamd.example.com <http://rspamd.example.com>* (where rspamd/redis
> >   runs)
> >
> > Each of these are FreeBSD 14.0p4.  Each has three IP addresses -- 2
> public
> > addresses (IPv4, IPv6) and a private (unroutable, IPv4) one.
> >
> > FreeBSD generates daily/weekly/monthly reports about the results of
> routine
> > system checks and mails them to root (so-called *periodic* reports).
>
> As you wrote in another mail you're sending those reports using
> (/usr/bin/mail),
> so mails are picked up locally by postix and are not injected via smtp.
>

<snip />

From rspamd at jubileegroup.co.uk  Sun Feb  4 10:53:09 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Sun, 4 Feb 2024 10:53:09 +0000 (GMT)
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <CALGB+_xQhb=Wkf7SPz9W8AO7jWSvzcWMLXPoJ8OM4hx7m5attw@mail.gmail.com>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
 <3586e42-ccd6-2641-fe92-b235468e46d@jubileegroup.co.uk>
 <CALGB+_xQhb=Wkf7SPz9W8AO7jWSvzcWMLXPoJ8OM4hx7m5attw@mail.gmail.com>
Message-ID: <751b4172-ba83-a380-2b2d-ae39a4f6e0c@jubileegroup.co.uk>

Hi there,

On Sat, 3 Feb 2024, Steve Witten wrote:
> On Sat, Feb 3, 2024 at 2:53?PM G.W. Haywood wrote:
> 
>> Please give the real names.
> 
> mail.niteflyte.net
> rspamd.niteflyte.net

Thanks.  I see that they have SPF records now too.  If you know the IP
address(es) of your sending mail server(s) it's much more efficient to
use the 'ip4:' and 'ip6:' mechanisms than 'a:', and especially 'mx:'.
You shouldn't use 'mx:' at all unless you really have to.  Once you're
happy with the record switch from '~all' to '-all' to show that you're
serious about it.  You'll be surprised how many criminals you'll find
forging mail from your domains.

> ...
> However, I think this is the answer:
>
>> ... the SPF record for example.com has nothing to do with the
>> (entirely separate) SPF records for mail.example.com and
>> rspamd.example.com ...
> 
> Maybe the better question to ask is: How can I prevent rspamd from
> scanning this mail?  Since it's internal, status-report kind of
> stuff, it's not really worthwhile to do this.

Maybe whitelisting?  With any email system there are many - perhaps
sometimes too many - ways to get things done.  If you're working with
very high volumes and performance may be an issue then you'd probably
want to arrange the filtering so that it was entirely skipped for any
local *known good* traffic.  This can mean rolling up your sleeves and
doing some serious digging, maybe some coding, and almost certainly
some documentation - so you can go back and fix it in a year's time.
For the much more numerous installations which handle modest volumes,
where the resource usage is not an issue, then keeping everything in
one place (the filter configuration is what I'd call "one place") is
probably simplest.

Here's a tutorial you might want to browse:

https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-scores/

The diagram in part 4 might be helpful.  I haven't checked how up-to-
date it all is.

Bear in mind that rspamd is capable of very high throughputs, so your
concern about scanning local known good mail might not be warranted.
Granted some virus scanners can be on the slow side but I personally
like to scan most outgoing mail.  Sort of belt && braces.  If I had to
deal with users of Windows boxes (thankfully I don't, any more) then
I'd insist on fully scanning all outgoing mail no matter what the cost.

-- 

73,
Ged.

From caponecicero at gmail.com  Sun Feb  4 16:03:18 2024
From: caponecicero at gmail.com (Steve Witten)
Date: Sun, 4 Feb 2024 08:03:18 -0800
Subject: [Rspamd-Users] I'm confused...
In-Reply-To: <751b4172-ba83-a380-2b2d-ae39a4f6e0c@jubileegroup.co.uk>
References: <CALGB+_xpLUB5o5sxam9aVi-CQ9i2Mj8pib0NcwxeQ0wu81pc3w@mail.gmail.com>
 <3586e42-ccd6-2641-fe92-b235468e46d@jubileegroup.co.uk>
 <CALGB+_xQhb=Wkf7SPz9W8AO7jWSvzcWMLXPoJ8OM4hx7m5attw@mail.gmail.com>
 <751b4172-ba83-a380-2b2d-ae39a4f6e0c@jubileegroup.co.uk>
Message-ID: <CALGB+_z0ZnRtyb3B4PU1MPhXSPj5-udAN2=JW8vKkvnYzxbRrg@mail.gmail.com>

See inline below...

On Sun, Feb 4, 2024 at 2:54?AM G.W. Haywood <rspamd at jubileegroup.co.uk>
wrote:

> Hi there,
>

<snip />

Thanks.  I see that they have SPF records now too.  If you know the IP
> address(es) of your sending mail server(s) it's much more efficient to
> use the 'ip4:' and 'ip6:' mechanisms than 'a:', and especially 'mx:'.
>

mail.niteflyte.net is a general purpose mail server.  However, it's for
family use so it only has 4 customers.  It runs on a really cheap VPS.  I
don't know a priori the ip addresses of all the senders.

However, the other two:  www.niteflyte.net (aka niteflyte.net) and
rspamd.niteflyte.net only send status & system health information to me.
The Dragonfly mail agents (dma) on those VPSs are send-only and the number
of addresses they know is two -- root and postmaster.

You shouldn't use 'mx:' at all unless you really have to.


I removed '+mx' in all of them.  Thanks for the tip.

Once you're happy with the record switch from '~all' to '-all' to show that
> you're
> serious about it.  You'll be surprised how many criminals you'll find
> forging mail from your domains.
>

Thanks for this tip as well.  I've done this in all of them too.


> > ...
> > However, I think this is the answer:
> >
> >> ... the SPF record for example.com has nothing to do with the
> >> (entirely separate) SPF records for mail.example.com and
> >> rspamd.example.com ...
> >
> > Maybe the better question to ask is: How can I prevent rspamd from
> > scanning this mail?  Since it's internal, status-report kind of
> > stuff, it's not really worthwhile to do this.
>
> Maybe whitelisting?  With any email system there are many - perhaps
> sometimes too many - ways to get things done.
>

I thought about this.  Most of the time, this status email is pretty *pro
forma* -- filled with lots of boilerplate.  It's kind of a nag to see it in
my inbox every day.  What I ultimately decided to do is send it all to
files in /var/log and rotate it automagically with newsyslogd.

Here's a tutorial you might want to browse:
>
>
> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-scores/
>
> The diagram in part 4 might be helpful.  I haven't checked how up-to-
> date it all is.
>

I've bookmarked it.  Thanks.

Thanks again for all your help.

Steve Witten

From caponecicero at gmail.com  Sun Feb  4 18:02:20 2024
From: caponecicero at gmail.com (Steve Witten)
Date: Sun, 4 Feb 2024 10:02:20 -0800
Subject: [Rspamd-Users] Proxy setup of rspamd web UI from another computer
Message-ID: <CALGB+_yiVFgpXeGhO6m-s4WXdDX2hfoC=gydaoieN9m5+ir9og@mail.gmail.com>

I have an apache web server at niteflyte.net.  I would like to use it to
proxy the rspamd web UI.

I tried this:

     ProxyRequests Off
     ProxyPass        /rspamd http://10.4.112.5:11334/
     ProxyPassReverse /rspamd http://10.4.112.5:11334/

This actually works but I don't get any of the Javascript goodies that make
the rspamd web UI work correctly.  Anyone have any ideas about how to do
this?

10.4.112.5 is a private network address for the rspamd machine.  rspamd
listens there.  The web server on rspamd.niteflyte.net proxies the web UI
this way.  The goal here is to eliminate that server.

Thanks in advance for your kind help.

Steve Witten

From list+rspamd at gcore.biz  Sun Feb  4 23:09:04 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Mon, 5 Feb 2024 00:09:04 +0100
Subject: [Rspamd-Users] Proxy setup of rspamd web UI from another
 computer
In-Reply-To: <CALGB+_yiVFgpXeGhO6m-s4WXdDX2hfoC=gydaoieN9m5+ir9og@mail.gmail.com>
References: <CALGB+_yiVFgpXeGhO6m-s4WXdDX2hfoC=gydaoieN9m5+ir9og@mail.gmail.com>
Message-ID: <75824658-6444-44AD-9E72-5BFB1C4DA681@gcore.biz>

> 
> I have an apache web server at niteflyte.net.  I would like to use it to
> proxy the rspamd web UI.
> 
> I tried this:
> 
>     ProxyRequests Off
>     ProxyPass        /rspamd http://10.4.112.5:11334/
>     ProxyPassReverse /rspamd http://10.4.112.5:11334/
> 
> This actually works but I don't get any of the Javascript goodies that make
> the rspamd web UI work correctly.  Anyone have any ideas about how to do
> this?

You could try to open your browser's developer tools and see if something
gets blocked (content-security policy comes to mind).

Best regards,
Gerald

From peter_rspamd at reinhold.dk  Mon Feb  5 07:17:26 2024
From: peter_rspamd at reinhold.dk (Peter Reinhold)
Date: Mon, 05 Feb 2024 08:17:26 +0100
Subject: [Rspamd-Users] Proxy setup of rspamd web UI from another
 computer
In-Reply-To: <CALGB+_yiVFgpXeGhO6m-s4WXdDX2hfoC=gydaoieN9m5+ir9og@mail.gmail.com>
References: <CALGB+_yiVFgpXeGhO6m-s4WXdDX2hfoC=gydaoieN9m5+ir9og@mail.gmail.com>
Message-ID: <1a583158e5de1f46c8800e5e744b9b22@reinhold.dk>

 From https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass

"If the first argument ends with a trailing /, the second argument 
should also end with a trailing /, and vice versa. Otherwise, the 
resulting requests to the backend may miss some needed slashes and do 
not deliver the expected results."

Try removing the trailing slash from your destination, or add one to 
your path, depending on what you are trying to do.

---
Peter Reinhold

On 2024-02-04 19:02, Steve Witten wrote:
> I have an apache web server at niteflyte.net.  I would like to use it 
> to
> proxy the rspamd web UI.
> 
> I tried this:
> 
>      ProxyRequests Off
>      ProxyPass        /rspamd http://10.4.112.5:11334/
>      ProxyPassReverse /rspamd http://10.4.112.5:11334/
> 
> This actually works but I don't get any of the Javascript goodies that 
> make
> the rspamd web UI work correctly.  Anyone have any ideas about how to 
> do
> this?
> 
> 10.4.112.5 is a private network address for the rspamd machine.  rspamd
> listens there.  The web server on rspamd.niteflyte.net proxies the web 
> UI
> this way.  The goal here is to eliminate that server.
> 
> Thanks in advance for your kind help.
> 
> Steve Witten

From rspamd at vlh.dk  Mon Feb  5 16:52:41 2024
From: rspamd at vlh.dk (rspamd at vlh.dk)
Date: Mon, 5 Feb 2024 17:52:41 +0100
Subject: [Rspamd-Users] Proxy setup of rspamd web UI from another
 computer
In-Reply-To: <75824658-6444-44AD-9E72-5BFB1C4DA681@gcore.biz>
References: <CALGB+_yiVFgpXeGhO6m-s4WXdDX2hfoC=gydaoieN9m5+ir9og@mail.gmail.com>
 <75824658-6444-44AD-9E72-5BFB1C4DA681@gcore.biz>
Message-ID: <009b01da5853$bc007f50$34017df0$@vlh.dk>

> >
> > I have an apache web server at niteflyte.net.  I would like to use it
> > to proxy the rspamd web UI.
> >
> > I tried this:
> >
> >     ProxyRequests Off
> >     ProxyPass        /rspamd http://10.4.112.5:11334/
> >     ProxyPassReverse /rspamd http://10.4.112.5:11334/
> >
> > This actually works but I don't get any of the Javascript goodies that
> > make the rspamd web UI work correctly.  Anyone have any ideas about
> > how to do this?
> 
> You could try to open your browser's developer tools and see if something
> gets blocked (content-security policy comes to mind).
> 
> Best regards,
> Gerald

For me it works with:

    ProxyPreserveHost On

    ProxyPass / http://10.0.0.2:11334/
    ProxyPassReverse / http://10.0.0.2:11334/


Regards,
Kim Sindalsen


From usenet at schani.com  Thu Feb  8 16:36:59 2024
From: usenet at schani.com (christian)
Date: Thu, 8 Feb 2024 17:36:59 +0100
Subject: [Rspamd-Users] Incorrect user rights
Message-ID: <782b3536-8991-4f70-a35c-1b16e1809e87@schani.com>

Hello,
I'm currently having the problem that rspamd and spamd spaassassin, 
which are integrated via the milter, cannot store bayes files.


2024-02-08T16:39:11.070520+01:00 wwl10 spamd[227936]: spamd: creating 
default_prefs: /root/.spamassassin/user_prefs
2024-02-08T16:39:11.070572+01:00 wwl10 spamd[227936]: spamd: failed to 
create readable default_prefs: /root/.spamassassin/user_prefs


RspamD uses _rspamd
and I set spamd to _rspamd

_rspamd   302408  0.8  0.4 331364 318660 ?       S    16:40   0:02 spamd 
child
_rspamd   302409  0.0  0.4 324864 309100 ?       S    16:40   0:00 spamd 
child

But it looks like I can't make the directory persist to /root/.spamassassin.
Where can I change the path?

I can't find a spamd.conf file

Thank you for your help
Christian

From usenet at schani.com  Thu Feb  8 17:32:19 2024
From: usenet at schani.com (christian)
Date: Thu, 8 Feb 2024 18:32:19 +0100
Subject: [Rspamd-Users] Search for terms in the body
Message-ID: <fc5fae40-5759-4afc-8b55-2898a41ef73e@schani.com>

Hello, is it possible to search for terms in the body with a multimap 
and add a counter to each term found? So count up the counter.

Thanks for help
Christian


	CONTENT_BLACKLISTED {
		type = "content";
		filter = "full"; # can be full, body, oneline, text, rawtext
		map = file://etc/rspamd/maps.d/regex_body.map;
		##prefilter = "true";
		##action = "add header";
		score = 1.0
		regexp = true;
		description = "Sucht nach W?rtern in der gesamten Email";
	}

From list+rspamd at gcore.biz  Thu Feb  8 17:45:11 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Thu, 8 Feb 2024 18:45:11 +0100
Subject: [Rspamd-Users] Incorrect user rights
In-Reply-To: <782b3536-8991-4f70-a35c-1b16e1809e87@schani.com>
References: <782b3536-8991-4f70-a35c-1b16e1809e87@schani.com>
Message-ID: <AACC52FE-45DB-4A2E-8B35-EEF2AE8D4FF9@gcore.biz>

> I'm currently having the problem that rspamd and spamd spaassassin, which are integrated via the milter, cannot store bayes files.
> 
> 
> 2024-02-08T16:39:11.070520+01:00 wwl10 spamd[227936]: spamd: creating default_prefs: /root/.spamassassin/user_prefs
> 2024-02-08T16:39:11.070572+01:00 wwl10 spamd[227936]: spamd: failed to create readable default_prefs: /root/.spamassassin/user_prefs

From a security perspective you do not want any software to process unchecked data received from the internet as root.

> RspamD uses _rspamd
> and I set spamd to _rspamd
> 
> _rspamd   302408  0.8  0.4 331364 318660 ?       S    16:40   0:02 spamd child
> _rspamd   302409  0.0  0.4 324864 309100 ?       S    16:40   0:00 spamd child
> 
> But it looks like I can't make the directory persist to /root/.spamassassin.
> Where can I change the path?
> 
> I can't find a spamd.conf file

Rspamd and SpamAssassin (spamd) are separate programs and are not related.
You might want to check SpamAssassin docs/mailinglists at https://spamassassin.apache.org/.
For running SpamAssassin it might also be a better option to go with amavisd-new: https://www.ijs.si/software/amavisd/

For migrating to rspamd it is possible to integrate SpamAssassin rules:
https://rspamd.com/doc/modules/spamassassin.html

Best regards,
Gerald

From list+rspamd at gcore.biz  Thu Feb  8 17:47:51 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Thu, 8 Feb 2024 18:47:51 +0100
Subject: [Rspamd-Users] Search for terms in the body
In-Reply-To: <fc5fae40-5759-4afc-8b55-2898a41ef73e@schani.com>
References: <fc5fae40-5759-4afc-8b55-2898a41ef73e@schani.com>
Message-ID: <538812F5-DE33-4751-A617-598A121A177F@gcore.biz>

> 
> Hello, is it possible to search for terms in the body with a multimap and add a counter to each term found? So count up the counter.
> 
> Thanks for help
> Christian
> 
> 
> 	CONTENT_BLACKLISTED {
> 		type = "content";
> 		filter = "full"; # can be full, body, oneline, text, rawtext
> 		map = file://etc/rspamd/maps.d/regex_body.map;
> 		##prefilter = "true";
> 		##action = "add header";
> 		score = 1.0
> 		regexp = true;
> 		description = "Sucht nach W?rtern in der gesamten Email";
> 	}

You can add multi = true; to CONTENT_BLACKLISTED, then every occurence will add to the spam score.

https://rspamd.com/doc/modules/multimap.html#get-all-matches

Best regards,
Gerald


From usenet at schani.com  Thu Feb  8 18:01:15 2024
From: usenet at schani.com (christian)
Date: Thu, 8 Feb 2024 19:01:15 +0100
Subject: [Rspamd-Users] Incorrect user rights
In-Reply-To: <AACC52FE-45DB-4A2E-8B35-EEF2AE8D4FF9@gcore.biz>
References: <782b3536-8991-4f70-a35c-1b16e1809e87@schani.com>
 <AACC52FE-45DB-4A2E-8B35-EEF2AE8D4FF9@gcore.biz>
Message-ID: <10ecd6e4-9022-40f9-876f-e296c105b19a@schani.com>

If I switch off spamassasin or spamd, will rspamd still perform a Bayes 
check on the email content?

Christian

Am 08.02.2024 um 18:45 schrieb Gerald Galster:
>> I'm currently having the problem that rspamd and spamd spaassassin, which are integrated via the milter, cannot store bayes files.
>>
>>
>> 2024-02-08T16:39:11.070520+01:00 wwl10 spamd[227936]: spamd: creating default_prefs: /root/.spamassassin/user_prefs
>> 2024-02-08T16:39:11.070572+01:00 wwl10 spamd[227936]: spamd: failed to create readable default_prefs: /root/.spamassassin/user_prefs
> 
>  From a security perspective you do not want any software to process unchecked data received from the internet as root.
> 
>> RspamD uses _rspamd
>> and I set spamd to _rspamd
>>
>> _rspamd   302408  0.8  0.4 331364 318660 ?       S    16:40   0:02 spamd child
>> _rspamd   302409  0.0  0.4 324864 309100 ?       S    16:40   0:00 spamd child
>>
>> But it looks like I can't make the directory persist to /root/.spamassassin.
>> Where can I change the path?
>>
>> I can't find a spamd.conf file
> 
> Rspamd and SpamAssassin (spamd) are separate programs and are not related.
> You might want to check SpamAssassin docs/mailinglists at https://spamassassin.apache.org/.
> For running SpamAssassin it might also be a better option to go with amavisd-new: https://www.ijs.si/software/amavisd/
> 
> For migrating to rspamd it is possible to integrate SpamAssassin rules:
> https://rspamd.com/doc/modules/spamassassin.html
> 
> Best regards,
> Gerald

From usenet at schani.com  Thu Feb  8 18:01:44 2024
From: usenet at schani.com (christian)
Date: Thu, 8 Feb 2024 19:01:44 +0100
Subject: [Rspamd-Users] Search for terms in the body
In-Reply-To: <538812F5-DE33-4751-A617-598A121A177F@gcore.biz>
References: <fc5fae40-5759-4afc-8b55-2898a41ef73e@schani.com>
 <538812F5-DE33-4751-A617-598A121A177F@gcore.biz>
Message-ID: <f7190963-ffec-4952-a070-6b3574ee29be@schani.com>

That's what I was looking for

Thanks

Am 08.02.2024 um 18:47 schrieb Gerald Galster:
>>
>> Hello, is it possible to search for terms in the body with a multimap and add a counter to each term found? So count up the counter.
>>
>> Thanks for help
>> Christian
>>
>>
>> 	CONTENT_BLACKLISTED {
>> 		type = "content";
>> 		filter = "full"; # can be full, body, oneline, text, rawtext
>> 		map = file://etc/rspamd/maps.d/regex_body.map;
>> 		##prefilter = "true";
>> 		##action = "add header";
>> 		score = 1.0
>> 		regexp = true;
>> 		description = "Sucht nach W?rtern in der gesamten Email";
>> 	}
> 
> You can add multi = true; to CONTENT_BLACKLISTED, then every occurence will add to the spam score.
> 
> https://rspamd.com/doc/modules/multimap.html#get-all-matches
> 
> Best regards,
> Gerald
> 

From list+rspamd at gcore.biz  Thu Feb  8 18:40:25 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Thu, 8 Feb 2024 19:40:25 +0100
Subject: [Rspamd-Users] Incorrect user rights
In-Reply-To: <10ecd6e4-9022-40f9-876f-e296c105b19a@schani.com>
References: <782b3536-8991-4f70-a35c-1b16e1809e87@schani.com>
 <AACC52FE-45DB-4A2E-8B35-EEF2AE8D4FF9@gcore.biz>
 <10ecd6e4-9022-40f9-876f-e296c105b19a@schani.com>
Message-ID: <DF1E44E6-29B9-4C35-B8C9-8A76F776DBFD@gcore.biz>

[As per convention post your reply below other answers please]

> If I switch off spamassasin or spamd, will rspamd still perform a Bayes check on the email content?

Rspamd includes a very capable bayesian filter:
https://rspamd.com/doc/configuration/statistic.html

If you have logging enabled you will see BAYES_SPAM and BAYES_HAM symbols/scores.

For bayes to work there needs to be a certain amount of spam/ham messages.
In case your rspamd is running for some time already there might be enough data,
e.g. blacklists that add to the score so that it's high enough to autolearn as spam.

https://rspamd.com/doc/configuration/statistic.html#autolearning

Otherwise you could train the bayes database:
https://rspamd.com/doc/faq.html#how-can-i-learn-messages

This is quite old, but nevertheless ...
https://rspamd.com/misc/2016/10/14/bayes-performance.html

So if you did not disable bayes filtering and if there is enough data available, it will work.

Best regards,
Gerald

From t.hendricks at interpool.de  Fri Feb  9 11:03:31 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Fri, 9 Feb 2024 12:03:31 +0100
Subject: [Rspamd-Users] Can't get DKIM signing to work
Message-ID: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>

Dear list!

Module is enabled:

rspamadm configdump -m
Modules enabled: rbl, ... dkim_signing, asn, ...

but this only reports

rspamadm configdump dkim_signing 
*** Section dkim_signing ***
sign_networks [
    "127.2.4.7",
]
Where does this IP address come from?

My /etc/rspamd/local.d/dkim_signing.conf:

sign_authenticated = true;
use_esld = true;
allow_username_mismatch = true;
use_domain = ?header";
allow_hdrfrom_mismatch = false;
allow_hdrfrom_mismatch_sign_networks = true;
selector="dkim";

# If false, messages with empty envelope from are not signed
allow_envfrom_empty = true;

# If true, multiple from headers are allowed (but only first is used)
allow_hdrfrom_multiple = false;

# Default path to key, can include '$domain' and '$selector' variables
path = "/var/lib/rspamd/dkim/$domain.$selector.key";

# Symbol to add when message is signed
symbol = "DKIM_SIGNED";

# If `true` get pubkey from DNS record and check if it matches private key
check_pubkey = false;
# Set to `false` if you want to skip signing if public and private keys mismatch
allow_pubkey_mismatch = true;

Log:
2024-02-09 11:41:53 #330848(normal) <7121d7>; task; dkim_symbol_callback: skip DKIM checks for local networks and authorized users
2024-02-09 11:41:53 #330848(normal) <7121d7>; lua; dmarc.lua:349: skip DMARC checks as either SPF or DKIM were not checked
2024-02-09 11:41:53 #330848(normal) <7121d7>; dkim_signing; lua_dkim_tools.lua:183: user is authenticated
2024-02-09 11:41:53 #330848(normal) <7121d7>; dkim_signing; lua_dkim_tools.lua:405: use domain(header) for signature: domain.tld
2024-02-09 11:41:53 #330848(normal) <7121d7>; dkim_signing; lua_dkim_tools.lua:425: final DKIM domain: domain.tld
2024-02-09 11:41:53 #330848(normal) <7121d7>; dkim_signing; lua_dkim_tools.lua:445: couldnt find domain in username

Why is it looking for the domain in username, despite: allow_username_mismatch = true;

What am I missing?

Thank you so much!

Tino


From rspamd-users at judo.za.org  Fri Feb  9 11:36:20 2024
From: rspamd-users at judo.za.org (Andrew Lewis)
Date: Fri, 09 Feb 2024 13:36:20 +0200
Subject: [Rspamd-Users] Can't get DKIM signing to work
In-Reply-To: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>
References: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>
Message-ID: <cae540d94ced61c0ba48ce3bd939c4973d4d1e70.camel@judo.za.org>

Hi Tino,

On Fri, 2024-02-09 at 12:03 +0100, Tino Hendricks wrote:
> Where does this IP address come from?

This is some curiosity from `cgp.inc`. It shouldn't rightfully exist
but it should neither hurt.

> My /etc/rspamd/local.d/dkim_signing.conf:

That you don't see this in configdump is suspicious; maybe the include
is missing in `modules.d/dkim_signing.conf` for some reason or you have
empty settings for this module in `rspamd.conf.override`.

> Why is it looking for the domain in username, despite:
> allow_username_mismatch = true;

The relevant setting is `use_domain` which you've apparently set though
it is quoted wrongly - besides that the setting is being lost.

Best,
-AL.

From rspamd at jubileegroup.co.uk  Fri Feb  9 11:36:30 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Fri, 9 Feb 2024 11:36:30 +0000 (GMT)
Subject: [Rspamd-Users] Can't get DKIM signing to work
In-Reply-To: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>
References: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>
Message-ID: <ca194d36-b6c0-adf5-8a34-296466e323@jubileegroup.co.uk>

Hi there,

On Fri, 9 Feb 2024, Tino Hendricks wrote:

> ...
> rspamadm configdump dkim_signing
> *** Section dkim_signing ***
> sign_networks [
>    "127.2.4.7",
> ]
> Where does this IP address come from?
> ...

Maybe here:

8<----------------------------------------------------------------------
$ grep -C10 -r 127.2.4.7 /etc/rspamd
/etc/rspamd/cgp.inc-# Please don't modify this file as your changes might be overwritten with
/etc/rspamd/cgp.inc-# the next update.
/etc/rspamd/cgp.inc-#
/etc/rspamd/cgp.inc-# This file defines some specific settings that are applicable merely when using
/etc/rspamd/cgp.inc-# CommuniGate Pro MTA and it's specific integration
/etc/rspamd/cgp.inc-
/etc/rspamd/cgp.inc-arc {
/etc/rspamd/cgp.inc:    sign_networks = [127.2.4.7];
/etc/rspamd/cgp.inc-}
/etc/rspamd/cgp.inc-
/etc/rspamd/cgp.inc-dkim_signing {
/etc/rspamd/cgp.inc:    sign_networks = [127.2.4.7];
/etc/rspamd/cgp.inc-}
/etc/rspamd/cgp.inc-
/etc/rspamd/cgp.inc-options {
/etc/rspamd/cgp.inc:    local_addrs = [127.2.4.7];
/etc/rspamd/cgp.inc-}
8<----------------------------------------------------------------------


> ...
> My /etc/rspamd/local.d/dkim_signing.conf:
> 
> sign_authenticated = true;
> use_esld = true;
> allow_username_mismatch = true;
> use_domain = ?header";

Is the crazy quote before 'header' intended?

Lastly, do these help?

https://github.com/rspamd/rspamd/issues/1593
https://github.com/rspamd/rspamd/issues/1768

-- 

73,
Ged.

From vsevolod at rspamd.com  Fri Feb  9 11:42:13 2024
From: vsevolod at rspamd.com (Vsevolod Stakhov)
Date: Fri, 9 Feb 2024 11:42:13 +0000
Subject: [Rspamd-Users] Can't get DKIM signing to work
In-Reply-To: <cae540d94ced61c0ba48ce3bd939c4973d4d1e70.camel@judo.za.org>
References: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>
 <cae540d94ced61c0ba48ce3bd939c4973d4d1e70.camel@judo.za.org>
Message-ID: <5872e481-adcd-f202-2290-499a17296ed6@rspamd.com>

On 09/02/2024 11:36, Andrew Lewis via Users wrote:
> Hi Tino,
> 
> On Fri, 2024-02-09 at 12:03 +0100, Tino Hendricks wrote:
>> Where does this IP address come from?
> 
> This is some curiosity from `cgp.inc`. It shouldn't rightfully exist
> but it should neither hurt.
> 
>> My /etc/rspamd/local.d/dkim_signing.conf:
> 
> That you don't see this in configdump is suspicious; maybe the include
> is missing in `modules.d/dkim_signing.conf` for some reason or you have
> empty settings for this module in `rspamd.conf.override`.
> 
>> Why is it looking for the domain in username, despite:
>> allow_username_mismatch = true;
> 
> The relevant setting is `use_domain` which you've apparently set though
> it is quoted wrongly - besides that the setting is being lost.


It's probably a good time to add tableshape schema for that? We should 
also think about usage of the annotated types that have been recent 
added to the tableshape.


From t.hendricks at interpool.de  Fri Feb  9 12:08:17 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Fri, 9 Feb 2024 13:08:17 +0100
Subject: [Rspamd-Users] Can't get DKIM signing to work
In-Reply-To: <5872e481-adcd-f202-2290-499a17296ed6@rspamd.com>
References: <3ADF202A-E37C-48DB-B5C6-3556BFFE3176@interpool.de>
 <cae540d94ced61c0ba48ce3bd939c4973d4d1e70.camel@judo.za.org>
 <5872e481-adcd-f202-2290-499a17296ed6@rspamd.com>
Message-ID: <C1DBE2BB-0648-430C-8E94-24C5307EE158@interpool.de>

From AL?s mail:

>> My /etc/rspamd/local.d/dkim_signing.conf:
>> sign_authenticated = true;
>> use_esld = true;
>> allow_username_mismatch = true;
>> use_domain = ?header";
> Is the crazy quote before 'header' intended?
"crazy quote? made my day! ;-D

I changed the copied config-text in my mail manually to resemble the last change of the server and my mail app changed the quotes to the designer ones.

> Am 09.02.2024 um 12:42 schrieb Vsevolod Stakhov <vsevolod at rspamd.com>:
> 
> On 09/02/2024 11:36, Andrew Lewis via Users wrote:
>> Hi Tino,
>> On Fri, 2024-02-09 at 12:03 +0100, Tino Hendricks wrote:
>>> Where does this IP address come from?
>> This is some curiosity from `cgp.inc`. It shouldn't rightfully exist
>> but it should neither hurt.
>>> My /etc/rspamd/local.d/dkim_signing.conf:
>> That you don't see this in configdump is suspicious; maybe the include
>> is missing in `modules.d/dkim_signing.conf` for some reason
Ha!
Accidentally I replaced the _modules.d_/dkim_signing.conf with my backup of local.d/dkim_signing.conf ????
Restoring the original modules.d/dkim_signing.conf of course makes it all work now.

Thanks, Vsevolod, for pointing me into the right direction.

Good day to you all!


>> or you have
>> empty settings for this module in `rspamd.conf.override`.
>>> Why is it looking for the domain in username, despite:
>>> allow_username_mismatch = true;
>> The relevant setting is `use_domain` which you've apparently set though
>> it is quoted wrongly - besides that the setting is being lost.
> 
> 
> It's probably a good time to add tableshape schema for that? We should also think about usage of the annotated types that have been recent added to the tableshape.
> 
> -- 
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users


From usenet at schani.com  Fri Feb  9 12:13:42 2024
From: usenet at schani.com (christian)
Date: Fri, 9 Feb 2024 13:13:42 +0100
Subject: [Rspamd-Users] Also a DKIM signing Question
Message-ID: <af8580db-d674-475f-b1d6-1261f0c1f5a0@schani.com>

Hello,
I also have a question about DKIM.
First of all, do you know the following service: 
https://www.learndmarc.com/ - that helped me a lot (no advertising).

My question:
I have a maindomain that runs Postfix and RspamD and is registered as an 
MX server in DNS. 100 domains send their emails via this domain.

user1 at domainA.com sends via maindomain.com
user2 at domainB.com sends via maindomain.com
user3 at domainC.com sends via maindomain.com
etc.

That works, but I can't get a working Dkim signature for the emails from 
user 3 at domainC.com, i.e. all non-masterdomain email addresses.

Do I have to enter a DNS entry and a server certificate for each domain 
even though they all send via the main domain.com?
Is RspamD's dkim_signing designed for this?
I would have to use the "xxx._domainkey TEXT "v=DKIM1\; for DNS every 
time. k=rsa\; p=MIIB ....", which is an enormous job.

Thanks for Help

Christian

From tacodewolff at gmail.com  Fri Feb  9 12:19:36 2024
From: tacodewolff at gmail.com (Taco de Wolff)
Date: Fri, 9 Feb 2024 09:19:36 -0300
Subject: [Rspamd-Users] Fwd: Prevent sender address spoofing envelope/header
 FROM
In-Reply-To: <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
Message-ID: <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>

Coming back to this question, instead of rejecting, perhaps we can correct
the user's mistake? I've noticed the DMARC munging module and it would
perhaps be a good idea to use (something like that) instead.

Reformulating the problem for outbound authenticated users: we have a user
that sends a mail using an envelope from address of noreply at domain.com but
specifies a header from address of someone at gmail.com (spoofing!). This
happened with a client that has a contact form that sets the header from
address to the e-mail address specified in the form, so that the client
receives mails as if they came from the person filling out the form (bad
configuration). This mail will succeed SPF and DKIM checks at the
recipient, but will fail DMARC since the header from address is not aligned
with SPF/DKIM (gmail.com != domain.com).

Can I use Rspamd to rewrite the header from address to be:
[header-from-address] via [envelope-from-address], only for outbound mail
and only when both domains differ?

Or should this be done in Postfix? I'm a little surprised it isn't a more
common problem to prevent spoofing on outbound mail...! How is everybody
else handling this?

Kind regards,
Taco de Wolff

PS: resending as I sent the original while unsubscribed

On Mon, Jan 22, 2024 at 3:57?PM Gerald Galster <list+rspamd at gcore.biz>
wrote:

> > Thanks Gerald, that's worth a try. I had another idea that might work and
> > wanted to check.
> >
> > While SPF verifies the envelope FROM address, and DKIM signs the message,
> > it is DMARC that enforces the header FROM address which makes it sent to
> > spam at the destination server. By default, Rspamd disables DMARC for
> > outgoing messages, what if we enable it so that it verifies DMARC locally
> > before sending out. This prevents it from getting to spam on the
> > destination server as it isn't sent out in the first place. Would that
> work?
>
> I don't think that will work. Dkim-signing means you trust the sender
> because of its ip or sasl authentification. That way you would revoke
> that trust. If a legitimate sender, that enforces dmarc for its domain,
> undergoes dmarc-checks before signing, that unsigned / to-be-signed mail
> would be rejected.
>
> Moreover checking dmarc in rspamd does not mean enforcing that policy,
> e.g. DMARC_POLICY_REJECT could just add to the spamlevel, not rejecting
> the mail immediately. To enforce that, something like
>
> actions = {
>   qarantine = "add_header";
>   reject = "reject";
> }
>
> had to be added to local.d/dmarc.conf.
>
> I would go with the lua example, extracting and comparing the
> lowercase envelope-sender/from domains for authenticated submitters.
>
> Best regards
> Gerald
> --
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users
>

From rspamd at jubileegroup.co.uk  Fri Feb  9 13:25:09 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Fri, 9 Feb 2024 13:25:09 +0000 (GMT)
Subject: [Rspamd-Users] Fwd: Prevent sender address spoofing
 envelope/header FROM
In-Reply-To: <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
 <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
Message-ID: <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>

Hi there,

On Fri, 9 Feb 2024, Taco de Wolff wrote:

> Coming back to this question, instead of rejecting, perhaps we can correct
> the user's mistake? I've noticed the DMARC munging module and it would
> perhaps be a good idea to use (something like that) instead.
>
> Reformulating the problem for outbound authenticated users: we have a user
> that sends a mail using an envelope from address of noreply at domain.com but
> specifies a header from address of someone at gmail.com (spoofing!). This
> happened with a client that has a contact form that sets the header from
> address to the e-mail address specified in the form, so that the client
> receives mails as if they came from the person filling out the form (bad
> configuration). ...

The 'From' field is intended to identify the author of the message.

https://datatracker.ietf.org/doc/html/rfc5322#section-3.6.2

The RFC doesn't specify what mechanisms will be used by the author to
create the message, only that the author is given in the 'From' field.

The way I read it, if somebody uses a Website to create an email and
that site causes this person's email address to appear in the 'From'
field of the email which it then sends, that is correct behaviour and
not 'spoofing'.

Whether it's sensible behaviour is another question.  If they can,
malicious users *will* deliberately use Web forms to send mail which
appears to be from addresses which they are not entitled to use.  This
is usually handled by a challenge-response mechanism in the Website;
before it sends mail FROM the user, the Website first sends mail TO
the user, and expects a response.  Absent the expected response, no
other mail is sent.  In this way malicious parties can't use random
email addresses for which they can't receive mail.

This may be a case for use of the 'Sender' field, where the Website is
in effect acting as the author's agent or secretary, and needs to have
its own mailbox - for example that of the Website's owner.

I understand that SPF/DKIM/DMARC are layered on top of all this, but
there's no point building on top of dodgy foundations.

-- 

73,
Ged.

From tacodewolff at gmail.com  Fri Feb  9 15:47:22 2024
From: tacodewolff at gmail.com (Taco de Wolff)
Date: Fri, 9 Feb 2024 12:47:22 -0300
Subject: [Rspamd-Users] Fwd: Prevent sender address spoofing
 envelope/header FROM
In-Reply-To: <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
 <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
 <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>
Message-ID: <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>

Thanks for the in-depth response. Looks like I lack understanding as I'm
not sure how I can make this work correctly.

I see your point in that the contact form is filled out by a customer and
is thus the appropriate header from address. The envelope from address is
the mail server, or the noreply at domain.com I created for this, since that
is where the mail is first sent from (the customer did not send a mail to
the website I could forward, it fills out a form which creates an email).

SPF checks out because the mail server's IP is allowed to send for the
given envelope from domain. DKIM checks out since it is signed using the
key for the envelope from domain and not altered on the way (if all is
well). DMARC will not check out since the header from address does not
align with neither SPF nor DKIM.

I don't see how I can make DMARC pass other than altering the header from
address. Does the sender address field help in passing DMARC? What am I
missing?

Kind regards,
Taco de Wolff


On Fri, Feb 9, 2024 at 10:27?AM G.W. Haywood <rspamd at jubileegroup.co.uk>
wrote:

> Hi there,
>
> On Fri, 9 Feb 2024, Taco de Wolff wrote:
>
> > Coming back to this question, instead of rejecting, perhaps we can
> correct
> > the user's mistake? I've noticed the DMARC munging module and it would
> > perhaps be a good idea to use (something like that) instead.
> >
> > Reformulating the problem for outbound authenticated users: we have a
> user
> > that sends a mail using an envelope from address of noreply at domain.com
> but
> > specifies a header from address of someone at gmail.com (spoofing!). This
> > happened with a client that has a contact form that sets the header from
> > address to the e-mail address specified in the form, so that the client
> > receives mails as if they came from the person filling out the form (bad
> > configuration). ...
>
> The 'From' field is intended to identify the author of the message.
>
> https://datatracker.ietf.org/doc/html/rfc5322#section-3.6.2
>
> The RFC doesn't specify what mechanisms will be used by the author to
> create the message, only that the author is given in the 'From' field.
>
> The way I read it, if somebody uses a Website to create an email and
> that site causes this person's email address to appear in the 'From'
> field of the email which it then sends, that is correct behaviour and
> not 'spoofing'.
>
> Whether it's sensible behaviour is another question.  If they can,
> malicious users *will* deliberately use Web forms to send mail which
> appears to be from addresses which they are not entitled to use.  This
> is usually handled by a challenge-response mechanism in the Website;
> before it sends mail FROM the user, the Website first sends mail TO
> the user, and expects a response.  Absent the expected response, no
> other mail is sent.  In this way malicious parties can't use random
> email addresses for which they can't receive mail.
>
> This may be a case for use of the 'Sender' field, where the Website is
> in effect acting as the author's agent or secretary, and needs to have
> its own mailbox - for example that of the Website's owner.
>
> I understand that SPF/DKIM/DMARC are layered on top of all this, but
> there's no point building on top of dodgy foundations.
>
> --
>
> 73,
> Ged.
> --
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users
>

From list+rspamd at gcore.biz  Fri Feb  9 17:29:25 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Fri, 9 Feb 2024 18:29:25 +0100
Subject: [Rspamd-Users] Also a DKIM signing Question
In-Reply-To: <af8580db-d674-475f-b1d6-1261f0c1f5a0@schani.com>
References: <af8580db-d674-475f-b1d6-1261f0c1f5a0@schani.com>
Message-ID: <E00B7DE4-90AB-4D5C-BB0D-D234D0FDFEC2@gcore.biz>

> My question:
> I have a maindomain that runs Postfix and RspamD and is registered as an MX server in DNS. 100 domains send their emails via this domain.

I'm not sure if I understood correctly, let's try this example:

Your domain is provider.com and all emails are received by mx1.provider.com.

In DNS speech: provider.com  IN MX 100 mx1.provider.com.

Then you have a few customers that use your mx1.provider.com for sending and receiving emails.

customer1.com IN MX 100 mx1.provider.com.
customer2.com IN MX 100 mx1.provider.com.
..

To enforce dmarc you would have to allow mx1.provider.com to send mails for customer1.com and customer2.com:

customer1.com IN TXT "v=spf1 mx ~all"
customer2.com IN TXT "v=spf1 mx ~all"

and/or DKIM sign mails for every customer domain:

selector._domainkey.customer1.com IN TXT "v=DKIM1..."
selector._domainkey.customer2.com IN TXT "v=DKIM1..."

Or to put it another way:
- with SPF you allow certain ips to send mails for certain domains
- with DKIM you permit correctly signed mails (ips do not matter)

Technically it would be possible to use the same DKIM-key for all customer*.com domains but that's not recommended.
It could be an enormous job to create an individual DKIM-key per domain, but if you're operating at scale it's time to automate.

Best regards,
Gerald

From rspamd at jubileegroup.co.uk  Fri Feb  9 18:01:56 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Fri, 9 Feb 2024 18:01:56 +0000 (GMT)
Subject: [Rspamd-Users] Also a DKIM signing Question
In-Reply-To: <af8580db-d674-475f-b1d6-1261f0c1f5a0@schani.com>
References: <af8580db-d674-475f-b1d6-1261f0c1f5a0@schani.com>
Message-ID: <6ba3edf8-85db-4017-8f21-139eb424a8c5@jubileegroup.co.uk>

Hi there,

I've made some assumptions below, please forgive me if they're wrong.
Please also forgive me my opinions. :)

On Fri, 9 Feb 2024, christian via Users wrote:

> do you know the following service:
> https://www.learndmarc.com/
> ...

I find I get more from reading something like

https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail

and maybe then something like

https://datatracker.ietf.org/doc/html/rfc6376

> I have a maindomain that runs Postfix and RspamD and is registered as an MX 
> server in DNS. 100 domains send their emails via this domain.

The precise meaning of "send their emails via this domain" isn't clear
to me.  Can you elaborate?  Please use the real names, not substitutes.

> user1 at domainA.com sends via maindomain.com
> user2 at domainB.com sends via maindomain.com
> user3 at domainC.com sends via maindomain.com
> etc.
>
> That works, but I can't get a working Dkim signature for the emails from user 
> 3 at domainC.com, i.e. all non-masterdomain email addresses.

You can only sign with a DKIM "Signing Domain Identifier" (SDID) which
you control, because you need to be able to put things like the public
keys in the DNS spaces of those domains.  That's kinda the whole point.

If domain[ABC].com are your customers' domains, presumably you don't
own or control them and you can't claim to have signed things for them.

They really ought to be able to sign mail for themselves, and then, if
you relay their mail for them, you'd just be relaying their signatures
along with any other headers in their mail.

> Do I have to enter a DNS entry and a server certificate for each domain ...

If you want to sign their mail on behalf of their domains then that is
as you point out going to involve you in some work, and responsibility.
For example you're going to have to be the custodian of their private
keys, and they're presumably going to look to you for the safe storage
and use of said private keys.  There's no way on Earth I'd trust *any*
third party with my private keys, it would be just asking for trouble.
They can't all use the *same* keys, because if they did they'd each be
able to sign mail on behalf of all the others and that could get ugly.

If I understand what you mean by 'the server certificate' then that
has nothing to do with DKIM.  The server's certificate is used when it
connects to some other server in order to verify that it is what it
claims to be (using mechanisms involving a trusted third party, which
are completely different from those used in DKIM).  All the receiving
server knows from the sending server's certificate is that it is what
it says it is - it gives no information about any mail messages (nor
anything else) which might subsequently pass between them.

But yes, each domain will normally need DNS entries for DKIM and yes,
it could be a bit of work.  If you are in fact going to perform this
function for a hundred domains then you'll probably want to have some
tooling to make it manageable.  At a guess you'd charge for it too.

You can sign messages using maindomain.com as the signing 'd=' domain
and, although this gives a recipient no information about whether or
not the server sending mail from user[123]@domain[ABC] is *entitled*
to send such mail, it does give some information about maindomain.com,
and the maindomain.com administrators can be held to account in case
there's some issue.  They can't deny they signed it - only they could,
unless they gave away their private keys (in itself a serious issue).

Being able to hold people to account is what started all this, so you
get a long way towards the objective with just that.  All that really
matters is that the signature is created by the entity which claims to
have created it and that it can be verified.

Within limits, signing things also means that if they were changed
after signing then the fact that they were changed can be detected by
a recipient by reading the signature, fetching from the DNS those
records which are needed to verify it, and doing the calculations.
That's a bonus, but although I have seen stories about it I have to
admit I've never personally seen it used for real.  When a signature
has failed verification it's always been either because something like
a mailing list screwed things up, or the signing was done wrongly in
the first place, or esle nobody cared and everyone just ignored it.
The limits that I mentioned mean that it's not always quite as simple
as it might first appear anyway.  Most of the time the real reason for
signing is just to get the mail delivered - and that isn't necessarily
what it's all about. :(

>...
> Is RspamD's dkim_signing designed for this?
> ...

I'm sorry, I don't use rspamd for signing mail and I'm not familiar
with the design criteria, but I'd expect it to be able to handle it.
Others here may be able to help you with that better than I can.

-- 

73,
Ged.

From rspamd at jubileegroup.co.uk  Fri Feb  9 18:14:55 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Fri, 9 Feb 2024 18:14:55 +0000 (GMT)
Subject: [Rspamd-Users] Fwd: Prevent sender address spoofing
 envelope/header FROM
In-Reply-To: <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
 <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
 <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>
 <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>
Message-ID: <86c8b6b3-a2dc-e7c-27db-341135e0713c@jubileegroup.co.uk>

Hi there,

On Fri, 9 Feb 2024, Taco de Wolff wrote:

> I see your point in that the contact form is filled out by a customer and
> is thus the appropriate header from address. The envelope from address is
> the mail server, or the noreply at domain.com I created for this, since that
> is where the mail is first sent from (the customer did not send a mail to
> the website I could forward, it fills out a form which creates an email).

Sure, that's what I'd thought you described.

> SPF checks out because the mail server's IP is allowed to send for the
> given envelope from domain.

ACK.

> DKIM checks out since it is signed using the key for the envelope
> from domain and not altered on the way (if all is well).

NAK.

DKIM does not use the envelope from address.  It calculates two
hashes, one on selected message headers and the other on the body.
The header fields selected MUST include the 'From' field.  Those
hashes are then used to create the signature, which just contains
these hashes and the signing parameters in encrypted form.

https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail#Overview
https://datatracker.ietf.org/doc/html/rfc5585#section-4.2

> DMARC will not check out since the header from address does not
> align with neither SPF nor DKIM.
> 
> I don't see how I can make DMARC pass other than altering the header from
> address. Does the sender address field help in passing DMARC? What am I
> missing?

No, the 'Sender' field isn't used in alignment tests.  And altering
things like the 'From' field is precisely what DKIM intends to detect!

If you control the envelope from, could you not set it to be the same
domain as is in the user's 'From' field?  Presumably the SPF records
for those domains already permit your server to send their mail, so
that SPF checks would still pass?  If not then you'll need a couple of
hundred new SPF records as well...

Maybe these will help:

https://en.wikipedia.org/wiki/DMARC#Alignment
https://superuser.com/questions/1427382/how-is-connected-envelop-from-and-mail-from-mail-header

-- 

73,
Ged.


From list+rspamd at gcore.biz  Fri Feb  9 18:55:55 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Fri, 9 Feb 2024 19:55:55 +0100
Subject: [Rspamd-Users] Prevent sender address spoofing envelope/header
 FROM
In-Reply-To: <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
 <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
 <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>
 <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>
Message-ID: <8EB4C913-8678-4E88-9108-0767AE168962@gcore.biz>

> I see your point in that the contact form is filled out by a customer and
> is thus the appropriate header from address. The envelope from address is

From my point of view the "author" is the contact form software, not the
customer. See it this way: you call a company and explain your problem.
The support agent opens a ticket and enters your request, including your
email address to keep you updated. In this case the support agent is the
author, respectively the contact form software.

In the long run you will attract spammers if you send emails/copies to
unverified addresses, even with captchas. Therefore, personally, I just
say thanks and that this request will be processed as soon as possible.

You provide a contact form so that others can contact you. A response
should originate from a legitimate address like "support at company.com",
not "noreply@". This way the contact form software can set legitimate
envelope/rfc5322 from addresses and eliminate all dmarc/dkim/spf problems.

[...]

> I don't see how I can make DMARC pass other than altering the header from
> address. Does the sender address field help in passing DMARC? What am I
> missing?

Just help your users to configure or choose a capable contact form software.

Best regards,
Gerald


From tacodewolff at gmail.com  Sat Feb 10 22:01:00 2024
From: tacodewolff at gmail.com (Taco de Wolff)
Date: Sat, 10 Feb 2024 19:01:00 -0300
Subject: [Rspamd-Users] Prevent sender address spoofing envelope/header
 FROM
In-Reply-To: <8EB4C913-8678-4E88-9108-0767AE168962@gcore.biz>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
 <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
 <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>
 <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>
 <8EB4C913-8678-4E88-9108-0767AE168962@gcore.biz>
Message-ID: <CABDg7L43nC3zTB_fpU_VmcLxBDM-BbDv7EZyg8YqcJCn2Jpn=w@mail.gmail.com>

Thanks for tuning in Gerald.

I agree that it is debatable from whom the mail is. The customer is not
sending an email after all, it is filling out a form. I can see your train
of thought Gerald and it is what I had in mind initially. Or see it this
way, the user supplies his email address in the contact form not to
indicate where he is sending _from_, but rather where he'd like to receive
follow-ups, e.g. a Reply-To address.

Additionally, I don't want to set the from address to whatever the user
supplies (such as yahoo or gmail) and pretend to be sending from that
domain. Beside it being easily abused, it is unfeasible to add all domains
on the internet to my SPF record. Other mail servers rightfully reject that
mail and I shouldn't add any of those domains to my SPF records anyways, as
I don't want to allow those servers to send email on my domain's behalf.

Not sure what you mean with a legitimate address though. I can create a
legitimate address such as noreply@ that only allows sending and not
receiving (and comply with spf/dkim/dmarc). I mean, it's a computer that is
sending the email, not a human that will check the inbox (much like the
emails generated by cron). Right?

I fully agree that helping to configure the software correctly is the first
step. However, I can only do so much as they can individually install other
WordPress plugins that override the defaults. Since clients will make it my
problem anyways, I was hoping to either reject sending (so to inform the
user quickly and to reduce sending invalid mail to keep up my IP
reputation) or correct it for them (change the header from address to
comply with spf).

Kind regards,
Taco de Wolff


On Fri, Feb 9, 2024 at 3:57?PM Gerald Galster <list+rspamd at gcore.biz> wrote:

> > I see your point in that the contact form is filled out by a customer and
> > is thus the appropriate header from address. The envelope from address is
>
> From my point of view the "author" is the contact form software, not the
> customer. See it this way: you call a company and explain your problem.
> The support agent opens a ticket and enters your request, including your
> email address to keep you updated. In this case the support agent is the
> author, respectively the contact form software.
>
> In the long run you will attract spammers if you send emails/copies to
> unverified addresses, even with captchas. Therefore, personally, I just
> say thanks and that this request will be processed as soon as possible.
>
> You provide a contact form so that others can contact you. A response
> should originate from a legitimate address like "support at company.com",
> not "noreply@". This way the contact form software can set legitimate
> envelope/rfc5322 from addresses and eliminate all dmarc/dkim/spf problems.
>
> [...]
>
> > I don't see how I can make DMARC pass other than altering the header from
> > address. Does the sender address field help in passing DMARC? What am I
> > missing?
>
> Just help your users to configure or choose a capable contact form
> software.
>
> Best regards,
> Gerald
>
> --
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users
>

From usenet at schani.com  Mon Feb 12 12:38:19 2024
From: usenet at schani.com (christian)
Date: Mon, 12 Feb 2024 13:38:19 +0100
Subject: [Rspamd-Users] Things I don't understand yet
Message-ID: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>

Hello,
With my RspamD filter things that don't explain occur to me.
I integrated a filter using multimap like this


	CONTENT_BLACKLISTED {
		type = "content";
		filter = "full";
		map = file://etc/rspamd/maps.d/regex_body-SPAM.map;
		##prefilter = "true";
		##action = "add header";
		score = 1.0
		multi = true;
		regexp = true;
		description = "Sucht nach SPAM W?rtern";
	}


After testing it works and it counts all the spam words and returns a 
bad rating.
But there are also incoming emails where such words appear but no 
evaluation is made based on my multimap entries. As if my multimap is 
not being respected. But this also happens in other cases. From time to 
time there are simply no rules displayed. Emails arrive and only have 
one filter rule displayed: "-4 replied".
Did I not understand something correctly or configure it correctly? Is 
this related to groups?
Thank you very much for help
Christian

From rspamd at jubileegroup.co.uk  Mon Feb 12 13:18:21 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Mon, 12 Feb 2024 13:18:21 +0000 (GMT)
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
Message-ID: <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>

Hi there,

On Mon, 12 Feb 2024, christian via Users wrote:

> ... no evaluation is made based on my multimap ...

Here's something I found useful to try to make sense of things:

https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/

Just a guess, it might be an entry triggered in a whitelist or for
example mid, both of which you probably have enabled.

Did you try

https://rspamd.com/doc/quickstart.html#the-rspamadm-command

rspamadm configtest

to see if it tells you anything interesting?

If none of that helps, maybe take a look at the entire config

rspamadm configdump

and you could even post it here.  Redact anything that you feel might
be sensitive.  I don't like to tell spammers what I'm looking for and
I'm sure some of them will be reading this list.

-- 

73,
Ged.

From list+rspamd at gcore.biz  Mon Feb 12 15:18:25 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Mon, 12 Feb 2024 16:18:25 +0100
Subject: [Rspamd-Users] Prevent sender address spoofing envelope/header
 FROM
In-Reply-To: <CABDg7L43nC3zTB_fpU_VmcLxBDM-BbDv7EZyg8YqcJCn2Jpn=w@mail.gmail.com>
References: <CABDg7L5M+S6fdAC+mpR4J1sQpiDqnBdDtAQ0BMqBwxbakOCGXA@mail.gmail.com>
 <21EF4008-1254-475B-B13A-F682EF5D31BF@gcore.biz>
 <CABDg7L6_ZMxGh6DZyY=pSWB-ODuO65cq4BCMLMJK94zdWNbeCw@mail.gmail.com>
 <40948A7F-A54C-4F20-9471-352E47AA7BF6@gcore.biz>
 <CABDg7L4qsOio=eBhS7cg3S5N-UeEpSMfFF=4Yrq66sbPD3h0Cg@mail.gmail.com>
 <CABDg7L5Miyx8z5R6SuA8XTGws=5zMzfvh83dBLwHj2yyovsGjA@mail.gmail.com>
 <3b34d088-7e75-d2e3-82df-bcb0c6986a5a@jubileegroup.co.uk>
 <CABDg7L5Q9TT3oQj9tqx-MAySc-tAQgmheqhgm=a5sP5pgrf9OQ@mail.gmail.com>
 <8EB4C913-8678-4E88-9108-0767AE168962@gcore.biz>
 <CABDg7L43nC3zTB_fpU_VmcLxBDM-BbDv7EZyg8YqcJCn2Jpn=w@mail.gmail.com>
Message-ID: <EBCC0327-6B0A-420E-A72C-FCAD06D9DD4E@gcore.biz>

> Not sure what you mean with a legitimate address though. I can create a
> legitimate address such as noreply@ that only allows sending and not
> receiving (and comply with spf/dkim/dmarc). I mean, it's a computer that is
> sending the email, not a human that will check the inbox (much like the
> emails generated by cron). Right?

Technically noreply@ is a valid address but as the name implies it's a dead
end, nobody will read that mail. From a customer service point of view it's
better to send with a real address like support at company.com, that makes it
easy for the customer to reach out or follow up. There are many companies
that send mails/newsletters with noreply@ to get their message over. They
take their customers' money but don't want to be bothered and make it hard
to reach out. I don't like that approach, but that's a personal preference.

> I fully agree that helping to configure the software correctly is the first
> step. However, I can only do so much as they can individually install other
> WordPress plugins that override the defaults. Since clients will make it my
> problem anyways, I was hoping to either reject sending (so to inform the
> user quickly and to reduce sending invalid mail to keep up my IP
> reputation) or correct it for them (change the header from address to
> comply with spf).

Typically the volume of such mails is low so that it won't harm your ip
reputation and I don't know of any companies that strictly enforce the
alignment of envelope/header from. Lots of shops are still sending with
"www-data@".

Just some ideas:

- monitor logs (maillog, phpmail.log)
- log additional info with postfix, e.g. header_checks: /^From:/  INFO
- write a postfix milter
- write a wrapper for /usr/sbin/sendmail that filters mails before submission
- enforce a custom php(-fpm) config (https://www.php.net/manual/en/mail.configuration.php)
- write a lua rule in rspamd that compares envelope/header from

Best regards,
Gerald

From usenet at schani.com  Mon Feb 12 20:52:48 2024
From: usenet at schani.com (christian)
Date: Mon, 12 Feb 2024 21:52:48 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
Message-ID: <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>

My rspamadm configtest gives the following:

rspamadm configtest
duplicate symbol: SUBJ_ALL_CAPS, skip registering
unknown type of attribute enabled for regexp module
cannot find dependency on symbol IS_IN_WHITELIST for symbol 
FORCE_ACTION_MY_WHITELIST
cannot find dependency on symbol FPROT_VIRUS for symbol 
FORCE_ACTION_MY_WHITELIST
syntax OK

But unfortunately I can't do anything with the information. What's more, 
I haven't changed anything in the symbols displayed.
Christian

Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
> Hi there,
> 
> On Mon, 12 Feb 2024, christian via Users wrote:
> 
>> ... no evaluation is made based on my multimap ...
> 
> Here's something I found useful to try to make sense of things:
> 
> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
> 
> Just a guess, it might be an entry triggered in a whitelist or for
> example mid, both of which you probably have enabled.
> 
> Did you try
> 
> https://rspamd.com/doc/quickstart.html#the-rspamadm-command
> 
> rspamadm configtest
> 
> to see if it tells you anything interesting?
> 
> If none of that helps, maybe take a look at the entire config
> 
> rspamadm configdump
> 
> and you could even post it here.? Redact anything that you feel might
> be sensitive.? I don't like to tell spammers what I'm looking for and
> I'm sure some of them will be reading this list.
> 

From usenet at schani.com  Mon Feb 12 21:15:38 2024
From: usenet at schani.com (christian)
Date: Mon, 12 Feb 2024 22:15:38 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
Message-ID: <22ff4502-5e99-49b8-ae16-64e7d0bb5aae@schani.com>

If I understand correctly, Rspamd's basic settings are fetched from 
/usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d. 
Some like multimap have no basic settings and are created themselves and 
read by rspamd. As a non-English speaker, I find it quite difficult to 
use the docs. Excuse me.
Christian

Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
> Hi there,
> 
> On Mon, 12 Feb 2024, christian via Users wrote:
> 
>> ... no evaluation is made based on my multimap ...
> 
> Here's something I found useful to try to make sense of things:
> 
> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
> 
> Just a guess, it might be an entry triggered in a whitelist or for
> example mid, both of which you probably have enabled.
> 
> Did you try
> 
> https://rspamd.com/doc/quickstart.html#the-rspamadm-command
> 
> rspamadm configtest
> 
> to see if it tells you anything interesting?
> 
> If none of that helps, maybe take a look at the entire config
> 
> rspamadm configdump
> 
> and you could even post it here.? Redact anything that you feel might
> be sensitive.? I don't like to tell spammers what I'm looking for and
> I'm sure some of them will be reading this list.
> 

From rspamd at jubileegroup.co.uk  Tue Feb 13 12:19:31 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Tue, 13 Feb 2024 12:19:31 +0000 (GMT)
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
Message-ID: <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>

Hello again,

On Mon, 12 Feb 2024, christian via Users wrote:
> Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
>> On Mon, 12 Feb 2024, christian via Users wrote:
>> 
>>> ... no evaluation is made based on my multimap ...
>> 
>> Here's something I found useful to try to make sense of things:
>> 
>> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
>> 
>> Just a guess, it might be an entry triggered in a whitelist or for
>> example mid, both of which you probably have enabled.
>> 
>> ... try ... rspamadm configtest
> 
> My rspamadm configtest gives the following:
>
> rspamadm configtest
> duplicate symbol: SUBJ_ALL_CAPS, skip registering
> unknown type of attribute enabled for regexp module
> cannot find dependency on symbol IS_IN_WHITELIST for symbol 
> FORCE_ACTION_MY_WHITELIST
> cannot find dependency on symbol FPROT_VIRUS for symbol 
> FORCE_ACTION_MY_WHITELIST
> syntax OK
>
> But unfortunately I can't do anything with the information.
> 
> What's more, I haven't changed anything in the symbols displayed.

With a configuration as complex as that of rspamd, in my view it's
absolutely essential to be very methodical when making any changes.

Do you have a record of all the changes that you made?  Do you have a
copy of your original configuration before you made any changes?  One
of the tools I use a great deal for this sort of work is 'diff'.  It
is, I suppose, really intended for people writing code, but it makes
it easy to compare huge sets of configuration options with changes in
files which hap-hazardly distributed within a large directory tree.

To test the latest installation instructions for rspamd which are
found on the rspamd Website, I followed them.  On a box which runs
Debian 'Bullseye', I ran the script below:

8<----------------------------------------------------------------------
#!/bin/bash
apt-get install -y lsb-release wget gpg
CODENAME=`lsb_release -c -s`
mkdir -p /etc/apt/keyrings
wget -O- https://rspamd.com/apt-stable/gpg.key | gpg --dearmor | tee /etc/apt/keyrings/rspamd.gpg > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main" | tee /etc/apt/sources.list.d/rspamd.list
echo "deb-src [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main"  | tee -a /etc/apt/sources.list.d/rspamd.list
apt-get update
apt-get --no-install-recommends install rspamd
8<----------------------------------------------------------------------

It only took a minute or so, and ran to completion with no issues.

The installation script is more or less according to the instructions at

https://rspamd.com/downloads.html

but I removed 'sudo' because I ran the script as root.

Below is what happened with rspamd newly installed using the script:

8<----------------------------------------------------------------------
# rspamadm --version
Rspamadm 3.8.1
# rspamadm configtest
syntax OK
# ls -l /usr/share/rspamd/
total 264
-rw-r--r--  1 root root 242327 Jan 25 19:10 effective_tld_names.dat
drwxr-xr-x  2 root root   4096 Feb 13 10:45 elastic
drwxr-xr-x  2 root root   4096 Feb 13 10:45 languages
drwxr-xr-x 10 root root   4096 Feb 13 10:45 lualib
drwxr-xr-x  2 root root   4096 Feb 13 10:45 plugins
drwxr-xr-x  4 root root   4096 Feb 13 10:45 rules
drwxr-xr-x  6 root root   4096 Feb 13 10:45 www
# ls -l /etc/rspamd/
total 96
-rw-r--r-- 1 root root 1213 Jan 25 19:10 actions.conf
-rw-r--r-- 1 root root  365 Jan 25 19:10 cgp.inc
-rw-r--r-- 1 root root 1318 Jan 25 19:10 common.conf
-rw-r--r-- 1 root root 7614 Jan 25 19:10 composites.conf
-rw-r--r-- 1 root root 5154 Jan 25 19:10 groups.conf
-rw-r--r-- 1 root root  874 Jan 25 19:10 lang_detection.inc
drwxr-xr-x 2 root root 4096 Jan 25 19:31 local.d
-rw-r--r-- 1 root root 1186 Jan 25 19:10 logging.inc
drwxr-xr-x 2 root root 4096 Feb 13 10:45 maps.d
-rw-r--r-- 1 root root  921 Jan 25 19:10 metrics.conf
-rw-r--r-- 1 root root  703 Jan 25 19:10 modules.conf
drwxr-xr-x 2 root root 4096 Feb 13 10:45 modules.d
-rw-r--r-- 1 root root 2020 Jan 25 19:10 options.inc
drwxr-xr-x 2 root root 4096 Jan 25 19:31 override.d
-rw-r--r-- 1 root root 2787 Jan 25 19:10 rspamd.conf
drwxr-xr-x 2 root root 4096 Feb 13 10:45 scores.d
-rw-r--r-- 1 root root 1799 Jan 25 19:10 settings.conf
-rw-r--r-- 1 root root 2169 Jan 25 19:10 statistic.conf
-rw-r--r-- 1 root root  618 Jan 25 19:10 worker-controller.inc
-rw-r--r-- 1 root root  654 Jan 25 19:10 worker-fuzzy.inc
-rw-r--r-- 1 root root  525 Jan 25 19:10 worker-normal.inc
-rw-r--r-- 1 root root 1363 Jan 25 19:10 worker-proxy.inc
8<----------------------------------------------------------------------

As you can see the files all have a relatively recent timestamp, which
I find comforting when I do a new installtaion.  Directory timestamps
in this case are when they were created, that's not important here but
if I have trouble finding changes in more or less anything I'll often
look for files/directories in a directory tree with recent timestamps.

It's clear that you've done things with which the rspamd configuration
test is a little unhappy, but it _does_ say it's 'OK' so I think it
will probably be working as designed.  Whether or not it's working as
you intend is another matter.  You could always try the stricter test
according to the 'man' page - this is the result on my new config:

# rspamadm --var=DBDIR=/tmp configtest -c /etc/rspamd/rspamd.conf -s
syntax OK

If it were my system I think I'd want to be sure that all the warnings
were gone before I'd be happy.  Perhaps you could back out the changes
which you've made one-by-one until the output is silenced.  If you do
not have a record of the changes you could start with a fresh config.
Document each and every change you make, but also (1) test it and (2)
dump it to a file each time you change it, with something like

# rspamadm configdump > ~/rspamd.configdump.$(date -Iseconds)

so you always have a record of a sane configuration if you break it.
Unfortunately comparing the outputs of 'configdump' taken at different
times might not be very helpful; the order in which the various parts
are output seems to be non-deterministic.

To address the issue of some mails being passed through without being
processed by some of the modules I think maybe you should investigate
the parts of the configuration which implement whitelisting, but *not*
before you're happy that you know exactly what changes you have made
to the configuration and exactly what you expect these changes to do.

If you still can't figure out why you aren't getting the results you
expect, you can post to this list the changes which you have made (or
as I said earlier the full configuration dump).

If you have an example mail which was blocked by your configuration,
(and should have been) and one which you feel is similar but was not
(and should have been) perhaps you can put them somewhere where we
can see them and try to do some guesswork.

> If I understand correctly, Rspamd's basic settings are fetched from
> /usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d.
> Some like multimap have no basic settings and are created themselves
> and read by rspamd.

Depending on how you look at things, fortunately or unfortunately the
paths for the configuration directories are themselves configurable.
That means the exact locations can be different in different systems
and the single biggest factors which determine the locations are the
way you installed rspamd and from where you downloaded the package.

If you installed as per the instruction on the rspamd Website your
understanding is correct.  I do not know what might happen if you
installed using packages from your Linux distribution, but if for
example you used the Debian package with just

apt-get install rspamd

there's a good chance that things will be four or five years out of
date and horribly broken.

> As a non-English speaker, I find it quite difficult to use the
> docs. Excuse me.

Your English is good but the documentation on the rspamd Website is I
fear not ideal for a new user.  That's why I pointed to an alternative
which, although it is somewhat out of date, tries a bit harder to make
the configuration make sense to someone new.  The array of options in
the rspamd configuration is immense and it's too easy to find yourself
lost in the woods.

-- 

73,
Ged.

From usenet at schani.com  Tue Feb 13 17:41:28 2024
From: usenet at schani.com (christian)
Date: Tue, 13 Feb 2024 18:41:28 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <715009BC-82B4-4B47-A071-DDB45F6B5BFA@interpool.de>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <22ff4502-5e99-49b8-ae16-64e7d0bb5aae@schani.com>
 <715009BC-82B4-4B47-A071-DDB45F6B5BFA@interpool.de>
Message-ID: <ae0e359d-a9af-4223-8ced-d91b3e3deb94@schani.com>

German version below

Hello Tino,

I think the bad documentation is because this is about ?nerd? software. 
There are only a few users there. And these users should know the 
subject anyway, or are fumbling their way into it. I'm one who has to 
fumble.

There is another spam filter that is poorly documented: ASSP 
http://www.thockar.com/assp-home/
I used ASSP for about 10 years and was very satisfied at the beginning. 
But the results and "false positives" kept increasing, and if you didn't 
keep your BLACK and WHITE lists meticulously clean, 50 spams could come 
through.

Six months ago there was new hardware again and I then took a closer 
look at Rspamd. I set up a test operation with 10 private email 
addresses for six months and trained Rspamd. Experience gained and tried 
out (SPF, DKIM, DMARC, CLAMD). The number of emails wasn't very large 
and I couldn't detect any spam outliers. So everything is great.

I've had around 800 email accounts running through it for 2 weeks now 
and the results are already better than with ASSP.

But, like many people who run a mail server, you sit in front of the log 
files (tail -f /var/log/mail.log) in the evening and watch what exactly 
happens. Then you notice some strange things.

Now I'm pursuing these "things" and learning a lot of new things about 
Rspamd.
I don't know yet whether the problems are with Rspamd or with me.
Probably with me ;-)
Some things are still not clear to me and the documentation doesn't help 
much. But through trial and error things progress slowly. I constantly 
check the results and they are very good, except for small details.

I've been helped a lot here on the mailing list. I hope I can give this 
back. But at the moment I'm still learning.

Greetings Christian




Hallo Tino,

ich glaube das mit der schlechten Dokumentation liegt daran das es hier 
um "Nerd" Software geht. Da gibt es nur wenige Nutzer. Und diese Nutzer 
sollten sich ohnehin mit der Materie auskennen, oder fummeln sich da 
rein. Ich bin einer der fummeln muss.

Es gibt noch einen Spamfilter der schlechter Dokumentiert ist: ASSP 
http://www.thockar.com/assp-home/
Ich habe ASSP ca. 10 Jahre benutzt und war am Anfang sehr zufrieden. 
Aber die Ergebnisse und "False Positive" wurden immer mehr, und wenn man 
seine BLACK und WHITE Listen nicht penibel sauber gehalten hat sind 
schon mal 50 Spams durch gerauscht.

Vor eine halben Jahr gab es mal wieder neue Hardware und habe mir dann 
Rspamd genauer angeschaut. Ein halbes Jahr einen Testbetrieb mit 10 
privaten Email Adressen angelegt und Rspamd angelernt. Erfahrungen 
gesammelt und ausprobiert (SPF,DKIM,DMARC,CLAMD). Die Email Anzahl war 
nicht sehr gro? und ich konnte auch keine Spam Ausrei?er erkennen. Also 
alles Super.

Seit 2 Wochen habe ich nun ca. 800 Email Konten dr?ber laufen und 
bereits jetzt sind die Ergebnisse besser als mit ASSP.

Aber, wie bei vielen die einen Mailserver betreiben, sitzt man dann 
Abends vor den Logfiles (tail -f /var/log/mail.log) und beobachtet was 
genau passiert. Dann fallen einem manche komischen Dinge auf.

Jetzt gehe ich diesen "Dingen" nach und lerne viel neues ?ber Rspamd.
Ich weis noch nicht ob die Probleme bei Rspamd liegen oder bei mir.
Vermutlich bei mir ;-)
Einiges ist mir noch nicht klar und die Dokumentation hilft nicht viel. 
Aber durch Try and Error gehts langsam voran. Ich kontrolliere die 
Ergebnisse st?ndig und sie sind sehr gut, bis auf Kleinigkeiten.

Hier in der Mailingliste wurde mir schon sehr geholfen. Ich hoffe ich 
kann das mal zur?ckgeben. Aber zur Zeit lerne ich noch.

Gr??e Christian

Am 13.02.2024 um 17:48 schrieb Tino Hendricks:
> Christian, Du sprichst mir aus der Seele! DANKE!
> 
> Ich finde auch die Docs echt schwierig. Ich halte mich f?r einen sehr 
> gut Englisch sprechenden und verstehenden Menschen (ich sach ma B2). Und 
> die Dinger sehen super aus.
> Aber der Aufbau ist einfach grottoid, finde ich. Es fehlt (mir) halt so 
> ne Grundstruktur, sowas, wie Du auch schreibst: ?Hey, Master-Config 
> steht in /usr/share/rspamd, kannste ?berschreiben mit??
> Und ?Was sind symbols, was sind groups, was sind modules, was sind 
> rules, was mach ich in JSON, was mache ich in lua??
> 
> Ich find?s mega schwierig, aus den super dargestellten Beispielen etwas 
> f?r mich abzuwandeln, weil ich nicht wei?, was f?r was ist. z.B. 
> https://rspamd.com/doc/configuration/composites.html 
> <https://rspamd.com/doc/configuration/composites.html>
> Wie aktiviert man das ein oder das andere?
> 
> Sch?n, dass ich nicht alleine bin.
> 
> Liebe Gr??e
> 
> Tino
> 
>> Am 12.02.2024 um 22:15 schrieb christian via Users 
>> <users at lists.rspamd.com>:
>>
>> If I understand correctly, Rspamd's basic settings are fetched from 
>> /usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d. 
>> Some like multimap have no basic settings and are created themselves 
>> and read by rspamd. As a non-English speaker, I find it quite 
>> difficult to use the docs. Excuse me.
>> Christian
>>
>> Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
>>> Hi there,
>>> On Mon, 12 Feb 2024, christian via Users wrote:
>>>> ... no evaluation is made based on my multimap ...
>>> Here's something I found useful to try to make sense of things:
>>> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
>>> Just a guess, it might be an entry triggered in a whitelist or for
>>> example mid, both of which you probably have enabled.
>>> Did you try
>>> https://rspamd.com/doc/quickstart.html#the-rspamadm-command
>>> rspamadm configtest
>>> to see if it tells you anything interesting?
>>> If none of that helps, maybe take a look at the entire config
>>> rspamadm configdump
>>> and you could even post it here.? Redact anything that you feel might
>>> be sensitive.? I don't like to tell spammers what I'm looking for and
>>> I'm sure some of them will be reading this list.
>> -- 
>> Users mailing list
>> Users at lists.rspamd.com
>> https://lists.rspamd.com/mailman/listinfo/users
> 

From usenet at schani.com  Thu Feb 15 11:16:50 2024
From: usenet at schani.com (christian)
Date: Thu, 15 Feb 2024 12:16:50 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
Message-ID: <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>

Hello G.W. Haywood,
Thank you for your detailed help. That gave me some food for thought.

During the night I completely removed Rspamd from my system and 
reinstalled it and also emptied redis. I have been using the Rspamd 
repository from the beginning as you described. So I'm up to date.
After reinstalling I have no errors through rspamadm configtest. Then I 
only set up the bare essentials. User account for the web frontend, 
Redis, ClamAV connection. And then I watched what happened for a day. 
Around 25,000 emails went through Rspamd and around 20 were ?false 
positives?. Which isn't actually that bad. It was just the usual 
suspects who always manage to get through.

Then I first attached my whitelist via multimap, as some important 
channels had values of +1-3 and were close to the spam limit.
Keep watching...
Then the blacklist. I was now able to pull out the usual suspects.
After 2-3 days now I notice that sometimes the multimap is not taken 
into account at all. So spam arrives which is then rated as 2 and gets 
through, but the header doesn't show me a test for my multimap blacklist 
that would have prevented it. I can't yet see a pattern when the 
multimap is ignored. Could it be that I first check a blacklist and then 
the whitelist? Is there even an order?
What I observe is that the number of checks that I display in the header 
of the email always varies and has no logic. Sometimes RBL's work and 
then only my blacklist and the RBL's don't. I don't see any connection yet.
But I will continue to monitor this.
Trie and error

Thanks
Christian



German Version

Hallo G.W. Haywood,
Danke f?r Deine ausf?hrliche Hilfe. Das hat mir einige Denkanst??e gegeben.

In der Nacht habe ich Rspamd komplett von meinem System entfernt und neu 
installiert und auch redis gelleert. Ich habe schon von Anfang an die 
Repository von Rspamd verwendet wie Du beschrieben hast. Ich bin also 
auf dem neusten Stand.
Nach der Neuinstallation habe ich durch rspamadm configtest keine 
Fehler. Dann habe ich nur das notwendigste eingerichtet. Benutzeraccount 
f?r das Webfrontend, Redis,ClamAV Verbindung. Und dann beobachtete ich 
einen Tag was passiert. Ca. 25000 Emails gingen durch Rspamd und ca. 20 
waren "false positive". Was eigentlich nicht so schlecht ist. Es waren 
nur die ?blichen verd?chtigen, die es immer schaffen durchzukommen.

Dann habe ich erst mal meine Whiteliste ?ber multimap eingeh?ngt, da 
einige wichtige sender werte von +1-3 hatten und knapp an der Spam 
Grenze waren.
Weiter beobachten ..
Dann die Blacklist. Die ?blichen verd?chtigen konnte ich jetzt auch 
rausziehen.
Nach jetzt 2-3 Tagen stelle ich doch fest, das manchmal die multimap gar 
nicht beachtet wird. Es kommt also Spam an die dann aber mit 2 bewertet 
wird und durchkommt, aber im Header wird mir kein Test auf meine 
multimap Blacklist angezeigt die es verhindert h?tte. Ich kann noch kein 
Muster erkennen, wann die multimap ignoriert wird. Kann es dran liegen 
das ich zuerst eine blacklist pr?fe und dann die whiteliste. Gibt es da 
?berhaupt eine Reihenfolge.
Was ich beobachte ist das die Anzahl der Pr?fungen, die ich mir im 
header der Mail anzeigen lasse, immer variiert und keine Logig hat. 
Manchmal greifen RBL?s und dann aber nur meine blacklist und die RBL?s 
nicht. Da sehe ich noch keinen Zusammenhang.
Ich werde das aber weiter beobachten.
Trie and Error

Danke
Christian


Am 13.02.2024 um 13:19 schrieb G.W. Haywood:
> Hello again,
> 
> On Mon, 12 Feb 2024, christian via Users wrote:
>> Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
>>> On Mon, 12 Feb 2024, christian via Users wrote:
>>>
>>>> ... no evaluation is made based on my multimap ...
>>>
>>> Here's something I found useful to try to make sense of things:
>>>
>>> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
>>>
>>> Just a guess, it might be an entry triggered in a whitelist or for
>>> example mid, both of which you probably have enabled.
>>>
>>> ... try ... rspamadm configtest
>>
>> My rspamadm configtest gives the following:
>>
>> rspamadm configtest
>> duplicate symbol: SUBJ_ALL_CAPS, skip registering
>> unknown type of attribute enabled for regexp module
>> cannot find dependency on symbol IS_IN_WHITELIST for symbol 
>> FORCE_ACTION_MY_WHITELIST
>> cannot find dependency on symbol FPROT_VIRUS for symbol 
>> FORCE_ACTION_MY_WHITELIST
>> syntax OK
>>
>> But unfortunately I can't do anything with the information.
>>
>> What's more, I haven't changed anything in the symbols displayed.
> 
> With a configuration as complex as that of rspamd, in my view it's
> absolutely essential to be very methodical when making any changes.
> 
> Do you have a record of all the changes that you made?? Do you have a
> copy of your original configuration before you made any changes?? One
> of the tools I use a great deal for this sort of work is 'diff'.? It
> is, I suppose, really intended for people writing code, but it makes
> it easy to compare huge sets of configuration options with changes in
> files which hap-hazardly distributed within a large directory tree.
> 
> To test the latest installation instructions for rspamd which are
> found on the rspamd Website, I followed them.? On a box which runs
> Debian 'Bullseye', I ran the script below:
> 
> 8<----------------------------------------------------------------------
> #!/bin/bash
> apt-get install -y lsb-release wget gpg
> CODENAME=`lsb_release -c -s`
> mkdir -p /etc/apt/keyrings
> wget -O- https://rspamd.com/apt-stable/gpg.key | gpg --dearmor | tee 
> /etc/apt/keyrings/rspamd.gpg > /dev/null
> echo "deb [signed-by=/etc/apt/keyrings/rspamd.gpg] 
> http://rspamd.com/apt-stable/ $CODENAME main" | tee 
> /etc/apt/sources.list.d/rspamd.list
> echo "deb-src [signed-by=/etc/apt/keyrings/rspamd.gpg] 
> http://rspamd.com/apt-stable/ $CODENAME main"? | tee -a 
> /etc/apt/sources.list.d/rspamd.list
> apt-get update
> apt-get --no-install-recommends install rspamd
> 8<----------------------------------------------------------------------
> 
> It only took a minute or so, and ran to completion with no issues.
> 
> The installation script is more or less according to the instructions at
> 
> https://rspamd.com/downloads.html
> 
> but I removed 'sudo' because I ran the script as root.
> 
> Below is what happened with rspamd newly installed using the script:
> 
> 8<----------------------------------------------------------------------
> # rspamadm --version
> Rspamadm 3.8.1
> # rspamadm configtest
> syntax OK
> # ls -l /usr/share/rspamd/
> total 264
> -rw-r--r--? 1 root root 242327 Jan 25 19:10 effective_tld_names.dat
> drwxr-xr-x? 2 root root?? 4096 Feb 13 10:45 elastic
> drwxr-xr-x? 2 root root?? 4096 Feb 13 10:45 languages
> drwxr-xr-x 10 root root?? 4096 Feb 13 10:45 lualib
> drwxr-xr-x? 2 root root?? 4096 Feb 13 10:45 plugins
> drwxr-xr-x? 4 root root?? 4096 Feb 13 10:45 rules
> drwxr-xr-x? 6 root root?? 4096 Feb 13 10:45 www
> # ls -l /etc/rspamd/
> total 96
> -rw-r--r-- 1 root root 1213 Jan 25 19:10 actions.conf
> -rw-r--r-- 1 root root? 365 Jan 25 19:10 cgp.inc
> -rw-r--r-- 1 root root 1318 Jan 25 19:10 common.conf
> -rw-r--r-- 1 root root 7614 Jan 25 19:10 composites.conf
> -rw-r--r-- 1 root root 5154 Jan 25 19:10 groups.conf
> -rw-r--r-- 1 root root? 874 Jan 25 19:10 lang_detection.inc
> drwxr-xr-x 2 root root 4096 Jan 25 19:31 local.d
> -rw-r--r-- 1 root root 1186 Jan 25 19:10 logging.inc
> drwxr-xr-x 2 root root 4096 Feb 13 10:45 maps.d
> -rw-r--r-- 1 root root? 921 Jan 25 19:10 metrics.conf
> -rw-r--r-- 1 root root? 703 Jan 25 19:10 modules.conf
> drwxr-xr-x 2 root root 4096 Feb 13 10:45 modules.d
> -rw-r--r-- 1 root root 2020 Jan 25 19:10 options.inc
> drwxr-xr-x 2 root root 4096 Jan 25 19:31 override.d
> -rw-r--r-- 1 root root 2787 Jan 25 19:10 rspamd.conf
> drwxr-xr-x 2 root root 4096 Feb 13 10:45 scores.d
> -rw-r--r-- 1 root root 1799 Jan 25 19:10 settings.conf
> -rw-r--r-- 1 root root 2169 Jan 25 19:10 statistic.conf
> -rw-r--r-- 1 root root? 618 Jan 25 19:10 worker-controller.inc
> -rw-r--r-- 1 root root? 654 Jan 25 19:10 worker-fuzzy.inc
> -rw-r--r-- 1 root root? 525 Jan 25 19:10 worker-normal.inc
> -rw-r--r-- 1 root root 1363 Jan 25 19:10 worker-proxy.inc
> 8<----------------------------------------------------------------------
> 
> As you can see the files all have a relatively recent timestamp, which
> I find comforting when I do a new installtaion.? Directory timestamps
> in this case are when they were created, that's not important here but
> if I have trouble finding changes in more or less anything I'll often
> look for files/directories in a directory tree with recent timestamps.
> 
> It's clear that you've done things with which the rspamd configuration
> test is a little unhappy, but it _does_ say it's 'OK' so I think it
> will probably be working as designed.? Whether or not it's working as
> you intend is another matter.? You could always try the stricter test
> according to the 'man' page - this is the result on my new config:
> 
> # rspamadm --var=DBDIR=/tmp configtest -c /etc/rspamd/rspamd.conf -s
> syntax OK
> 
> If it were my system I think I'd want to be sure that all the warnings
> were gone before I'd be happy.? Perhaps you could back out the changes
> which you've made one-by-one until the output is silenced.? If you do
> not have a record of the changes you could start with a fresh config.
> Document each and every change you make, but also (1) test it and (2)
> dump it to a file each time you change it, with something like
> 
> # rspamadm configdump > ~/rspamd.configdump.$(date -Iseconds)
> 
> so you always have a record of a sane configuration if you break it.
> Unfortunately comparing the outputs of 'configdump' taken at different
> times might not be very helpful; the order in which the various parts
> are output seems to be non-deterministic.
> 
> To address the issue of some mails being passed through without being
> processed by some of the modules I think maybe you should investigate
> the parts of the configuration which implement whitelisting, but *not*
> before you're happy that you know exactly what changes you have made
> to the configuration and exactly what you expect these changes to do.
> 
> If you still can't figure out why you aren't getting the results you
> expect, you can post to this list the changes which you have made (or
> as I said earlier the full configuration dump).
> 
> If you have an example mail which was blocked by your configuration,
> (and should have been) and one which you feel is similar but was not
> (and should have been) perhaps you can put them somewhere where we
> can see them and try to do some guesswork.
> 
>> If I understand correctly, Rspamd's basic settings are fetched from
>> /usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d.
>> Some like multimap have no basic settings and are created themselves
>> and read by rspamd.
> 
> Depending on how you look at things, fortunately or unfortunately the
> paths for the configuration directories are themselves configurable.
> That means the exact locations can be different in different systems
> and the single biggest factors which determine the locations are the
> way you installed rspamd and from where you downloaded the package.
> 
> If you installed as per the instruction on the rspamd Website your
> understanding is correct.? I do not know what might happen if you
> installed using packages from your Linux distribution, but if for
> example you used the Debian package with just
> 
> apt-get install rspamd
> 
> there's a good chance that things will be four or five years out of
> date and horribly broken.
> 
>> As a non-English speaker, I find it quite difficult to use the
>> docs. Excuse me.
> 
> Your English is good but the documentation on the rspamd Website is I
> fear not ideal for a new user.? That's why I pointed to an alternative
> which, although it is somewhat out of date, tries a bit harder to make
> the configuration make sense to someone new.? The array of options in
> the rspamd configuration is immense and it's too easy to find yourself
> lost in the woods.
> 

From info at hansvaneijsden.nl  Thu Feb 15 15:55:39 2024
From: info at hansvaneijsden.nl (Hans van Eijsden)
Date: Thu, 15 Feb 2024 16:55:39 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
Message-ID: <8BC7885C-22BF-49C3-82BE-6B68EEF7638C@hansvaneijsden.nl>

Hi Christian,

Some years ago, I encountered a similar situation, and I found a solution that might help you:

- I created a file named "options.inc" in the directory "/etc/rspamd/local.d/"
- Inside this file, I added the line: "check_all_filters = true;"

According to the documentation at https://rspamd.com/doc/configuration/options.html, this setting by default (set to "false") disables optimizations when a message exceeds the reject score for the default metric.

I believe implementing this change by setting it to "true" should address the issue you mentioned, particularly regarding the occasional skipping of the multimap.

Best regards,

Hans van Eijsden

> Op 15 feb 2024, om 12:16 heeft christian via Users <users at lists.rspamd.com> het volgende geschreven:
> 
> Hello G.W. Haywood,
> Thank you for your detailed help. That gave me some food for thought.
> 
> During the night I completely removed Rspamd from my system and reinstalled it and also emptied redis. I have been using the Rspamd repository from the beginning as you described. So I'm up to date.
> After reinstalling I have no errors through rspamadm configtest. Then I only set up the bare essentials. User account for the web frontend, Redis, ClamAV connection. And then I watched what happened for a day. Around 25,000 emails went through Rspamd and around 20 were ?false positives?. Which isn't actually that bad. It was just the usual suspects who always manage to get through.
> 
> Then I first attached my whitelist via multimap, as some important channels had values of +1-3 and were close to the spam limit.
> Keep watching...
> Then the blacklist. I was now able to pull out the usual suspects.
> After 2-3 days now I notice that sometimes the multimap is not taken into account at all. So spam arrives which is then rated as 2 and gets through, but the header doesn't show me a test for my multimap blacklist that would have prevented it. I can't yet see a pattern when the multimap is ignored. Could it be that I first check a blacklist and then the whitelist? Is there even an order?
> What I observe is that the number of checks that I display in the header of the email always varies and has no logic. Sometimes RBL's work and then only my blacklist and the RBL's don't. I don't see any connection yet.
> But I will continue to monitor this.
> Trie and error
> 
> Thanks
> Christian
> 
> 
> 
> German Version
> 
> Hallo G.W. Haywood,
> Danke f?r Deine ausf?hrliche Hilfe. Das hat mir einige Denkanst??e gegeben.
> 
> In der Nacht habe ich Rspamd komplett von meinem System entfernt und neu installiert und auch redis gelleert. Ich habe schon von Anfang an die Repository von Rspamd verwendet wie Du beschrieben hast. Ich bin also auf dem neusten Stand.
> Nach der Neuinstallation habe ich durch rspamadm configtest keine Fehler. Dann habe ich nur das notwendigste eingerichtet. Benutzeraccount f?r das Webfrontend, Redis,ClamAV Verbindung. Und dann beobachtete ich einen Tag was passiert. Ca. 25000 Emails gingen durch Rspamd und ca. 20 waren "false positive". Was eigentlich nicht so schlecht ist. Es waren nur die ?blichen verd?chtigen, die es immer schaffen durchzukommen.
> 
> Dann habe ich erst mal meine Whiteliste ?ber multimap eingeh?ngt, da einige wichtige sender werte von +1-3 hatten und knapp an der Spam Grenze waren.
> Weiter beobachten ..
> Dann die Blacklist. Die ?blichen verd?chtigen konnte ich jetzt auch rausziehen.
> Nach jetzt 2-3 Tagen stelle ich doch fest, das manchmal die multimap gar nicht beachtet wird. Es kommt also Spam an die dann aber mit 2 bewertet wird und durchkommt, aber im Header wird mir kein Test auf meine multimap Blacklist angezeigt die es verhindert h?tte. Ich kann noch kein Muster erkennen, wann die multimap ignoriert wird. Kann es dran liegen das ich zuerst eine blacklist pr?fe und dann die whiteliste. Gibt es da ?berhaupt eine Reihenfolge.
> Was ich beobachte ist das die Anzahl der Pr?fungen, die ich mir im header der Mail anzeigen lasse, immer variiert und keine Logig hat. Manchmal greifen RBL?s und dann aber nur meine blacklist und die RBL?s nicht. Da sehe ich noch keinen Zusammenhang.
> Ich werde das aber weiter beobachten.
> Trie and Error
> 
> Danke
> Christian
> 
> 
> Am 13.02.2024 um 13:19 schrieb G.W. Haywood:
>> Hello again,
>> On Mon, 12 Feb 2024, christian via Users wrote:
>>> Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
>>>> On Mon, 12 Feb 2024, christian via Users wrote:
>>>> 
>>>>> ... no evaluation is made based on my multimap ...
>>>> 
>>>> Here's something I found useful to try to make sense of things:
>>>> 
>>>> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
>>>> 
>>>> Just a guess, it might be an entry triggered in a whitelist or for
>>>> example mid, both of which you probably have enabled.
>>>> 
>>>> ... try ... rspamadm configtest
>>> 
>>> My rspamadm configtest gives the following:
>>> 
>>> rspamadm configtest
>>> duplicate symbol: SUBJ_ALL_CAPS, skip registering
>>> unknown type of attribute enabled for regexp module
>>> cannot find dependency on symbol IS_IN_WHITELIST for symbol FORCE_ACTION_MY_WHITELIST
>>> cannot find dependency on symbol FPROT_VIRUS for symbol FORCE_ACTION_MY_WHITELIST
>>> syntax OK
>>> 
>>> But unfortunately I can't do anything with the information.
>>> 
>>> What's more, I haven't changed anything in the symbols displayed.
>> With a configuration as complex as that of rspamd, in my view it's
>> absolutely essential to be very methodical when making any changes.
>> Do you have a record of all the changes that you made?  Do you have a
>> copy of your original configuration before you made any changes?  One
>> of the tools I use a great deal for this sort of work is 'diff'.  It
>> is, I suppose, really intended for people writing code, but it makes
>> it easy to compare huge sets of configuration options with changes in
>> files which hap-hazardly distributed within a large directory tree.
>> To test the latest installation instructions for rspamd which are
>> found on the rspamd Website, I followed them.  On a box which runs
>> Debian 'Bullseye', I ran the script below:
>> 8<----------------------------------------------------------------------
>> #!/bin/bash
>> apt-get install -y lsb-release wget gpg
>> CODENAME=`lsb_release -c -s`
>> mkdir -p /etc/apt/keyrings
>> wget -O- https://rspamd.com/apt-stable/gpg.key | gpg --dearmor | tee /etc/apt/keyrings/rspamd.gpg > /dev/null
>> echo "deb [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main" | tee /etc/apt/sources.list.d/rspamd.list
>> echo "deb-src [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main"  | tee -a /etc/apt/sources.list.d/rspamd.list
>> apt-get update
>> apt-get --no-install-recommends install rspamd
>> 8<----------------------------------------------------------------------
>> It only took a minute or so, and ran to completion with no issues.
>> The installation script is more or less according to the instructions at
>> https://rspamd.com/downloads.html
>> but I removed 'sudo' because I ran the script as root.
>> Below is what happened with rspamd newly installed using the script:
>> 8<----------------------------------------------------------------------
>> # rspamadm --version
>> Rspamadm 3.8.1
>> # rspamadm configtest
>> syntax OK
>> # ls -l /usr/share/rspamd/
>> total 264
>> -rw-r--r--  1 root root 242327 Jan 25 19:10 effective_tld_names.dat
>> drwxr-xr-x  2 root root   4096 Feb 13 10:45 elastic
>> drwxr-xr-x  2 root root   4096 Feb 13 10:45 languages
>> drwxr-xr-x 10 root root   4096 Feb 13 10:45 lualib
>> drwxr-xr-x  2 root root   4096 Feb 13 10:45 plugins
>> drwxr-xr-x  4 root root   4096 Feb 13 10:45 rules
>> drwxr-xr-x  6 root root   4096 Feb 13 10:45 www
>> # ls -l /etc/rspamd/
>> total 96
>> -rw-r--r-- 1 root root 1213 Jan 25 19:10 actions.conf
>> -rw-r--r-- 1 root root  365 Jan 25 19:10 cgp.inc
>> -rw-r--r-- 1 root root 1318 Jan 25 19:10 common.conf
>> -rw-r--r-- 1 root root 7614 Jan 25 19:10 composites.conf
>> -rw-r--r-- 1 root root 5154 Jan 25 19:10 groups.conf
>> -rw-r--r-- 1 root root  874 Jan 25 19:10 lang_detection.inc
>> drwxr-xr-x 2 root root 4096 Jan 25 19:31 local.d
>> -rw-r--r-- 1 root root 1186 Jan 25 19:10 logging.inc
>> drwxr-xr-x 2 root root 4096 Feb 13 10:45 maps.d
>> -rw-r--r-- 1 root root  921 Jan 25 19:10 metrics.conf
>> -rw-r--r-- 1 root root  703 Jan 25 19:10 modules.conf
>> drwxr-xr-x 2 root root 4096 Feb 13 10:45 modules.d
>> -rw-r--r-- 1 root root 2020 Jan 25 19:10 options.inc
>> drwxr-xr-x 2 root root 4096 Jan 25 19:31 override.d
>> -rw-r--r-- 1 root root 2787 Jan 25 19:10 rspamd.conf
>> drwxr-xr-x 2 root root 4096 Feb 13 10:45 scores.d
>> -rw-r--r-- 1 root root 1799 Jan 25 19:10 settings.conf
>> -rw-r--r-- 1 root root 2169 Jan 25 19:10 statistic.conf
>> -rw-r--r-- 1 root root  618 Jan 25 19:10 worker-controller.inc
>> -rw-r--r-- 1 root root  654 Jan 25 19:10 worker-fuzzy.inc
>> -rw-r--r-- 1 root root  525 Jan 25 19:10 worker-normal.inc
>> -rw-r--r-- 1 root root 1363 Jan 25 19:10 worker-proxy.inc
>> 8<----------------------------------------------------------------------
>> As you can see the files all have a relatively recent timestamp, which
>> I find comforting when I do a new installtaion.  Directory timestamps
>> in this case are when they were created, that's not important here but
>> if I have trouble finding changes in more or less anything I'll often
>> look for files/directories in a directory tree with recent timestamps.
>> It's clear that you've done things with which the rspamd configuration
>> test is a little unhappy, but it _does_ say it's 'OK' so I think it
>> will probably be working as designed.  Whether or not it's working as
>> you intend is another matter.  You could always try the stricter test
>> according to the 'man' page - this is the result on my new config:
>> # rspamadm --var=DBDIR=/tmp configtest -c /etc/rspamd/rspamd.conf -s
>> syntax OK
>> If it were my system I think I'd want to be sure that all the warnings
>> were gone before I'd be happy.  Perhaps you could back out the changes
>> which you've made one-by-one until the output is silenced.  If you do
>> not have a record of the changes you could start with a fresh config.
>> Document each and every change you make, but also (1) test it and (2)
>> dump it to a file each time you change it, with something like
>> # rspamadm configdump > ~/rspamd.configdump.$(date -Iseconds)
>> so you always have a record of a sane configuration if you break it.
>> Unfortunately comparing the outputs of 'configdump' taken at different
>> times might not be very helpful; the order in which the various parts
>> are output seems to be non-deterministic.
>> To address the issue of some mails being passed through without being
>> processed by some of the modules I think maybe you should investigate
>> the parts of the configuration which implement whitelisting, but *not*
>> before you're happy that you know exactly what changes you have made
>> to the configuration and exactly what you expect these changes to do.
>> If you still can't figure out why you aren't getting the results you
>> expect, you can post to this list the changes which you have made (or
>> as I said earlier the full configuration dump).
>> If you have an example mail which was blocked by your configuration,
>> (and should have been) and one which you feel is similar but was not
>> (and should have been) perhaps you can put them somewhere where we
>> can see them and try to do some guesswork.
>>> If I understand correctly, Rspamd's basic settings are fetched from
>>> /usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d.
>>> Some like multimap have no basic settings and are created themselves
>>> and read by rspamd.
>> Depending on how you look at things, fortunately or unfortunately the
>> paths for the configuration directories are themselves configurable.
>> That means the exact locations can be different in different systems
>> and the single biggest factors which determine the locations are the
>> way you installed rspamd and from where you downloaded the package.
>> If you installed as per the instruction on the rspamd Website your
>> understanding is correct.  I do not know what might happen if you
>> installed using packages from your Linux distribution, but if for
>> example you used the Debian package with just
>> apt-get install rspamd
>> there's a good chance that things will be four or five years out of
>> date and horribly broken.
>>> As a non-English speaker, I find it quite difficult to use the
>>> docs. Excuse me.
>> Your English is good but the documentation on the rspamd Website is I
>> fear not ideal for a new user.  That's why I pointed to an alternative
>> which, although it is somewhat out of date, tries a bit harder to make
>> the configuration make sense to someone new.  The array of options in
>> the rspamd configuration is immense and it's too easy to find yourself
>> lost in the woods.
> -- 
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4335 bytes
Desc: not available
URL: <https://lists.rspamd.com/pipermail/users/attachments/20240215/0517d9ca/attachment.bin>

From info at hansvaneijsden.nl  Thu Feb 15 16:01:55 2024
From: info at hansvaneijsden.nl (Hans van Eijsden)
Date: Thu, 15 Feb 2024 17:01:55 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <8BC7885C-22BF-49C3-82BE-6B68EEF7638C@hansvaneijsden.nl>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <8BC7885C-22BF-49C3-82BE-6B68EEF7638C@hansvaneijsden.nl>
Message-ID: <7B1FD5F4-4E75-41BF-B44E-9B2D74CE4542@hansvaneijsden.nl>

Hi Christian,

A correction to my previous reply:

> According to the documentation at https://rspamd.com/doc/configuration/options.html, this setting by default (set to "false") disables optimizations when a message exceeds the reject score for the default metric.

According to the documentation at https://rspamd.com/doc/configuration/options.html, this setting by default (set to "false") enables optimizations when a message exceeds the reject score for the default metric. Setting it to "true" disables those optimizations: it then checks all filters anytime.

Best regards / Met vriendelijke groet,

Hans van Eijsden


> Op 15 feb 2024, om 16:55 heeft Hans van Eijsden <info at hansvaneijsden.nl> het volgende geschreven:
> 
> Hi Christian,
> 
> Some years ago, I encountered a similar situation, and I found a solution that might help you:
> 
> - I created a file named "options.inc" in the directory "/etc/rspamd/local.d/"
> - Inside this file, I added the line: "check_all_filters = true;"
> 
> According to the documentation at https://rspamd.com/doc/configuration/options.html, this setting by default (set to "false") disables optimizations when a message exceeds the reject score for the default metric.
> 
> I believe implementing this change by setting it to "true" should address the issue you mentioned, particularly regarding the occasional skipping of the multimap.
> 
> Best regards,
> 
> Hans van Eijsden
> 
>> Op 15 feb 2024, om 12:16 heeft christian via Users <users at lists.rspamd.com> het volgende geschreven:
>> 
>> Hello G.W. Haywood,
>> Thank you for your detailed help. That gave me some food for thought.
>> 
>> During the night I completely removed Rspamd from my system and reinstalled it and also emptied redis. I have been using the Rspamd repository from the beginning as you described. So I'm up to date.
>> After reinstalling I have no errors through rspamadm configtest. Then I only set up the bare essentials. User account for the web frontend, Redis, ClamAV connection. And then I watched what happened for a day. Around 25,000 emails went through Rspamd and around 20 were ?false positives?. Which isn't actually that bad. It was just the usual suspects who always manage to get through.
>> 
>> Then I first attached my whitelist via multimap, as some important channels had values of +1-3 and were close to the spam limit.
>> Keep watching...
>> Then the blacklist. I was now able to pull out the usual suspects.
>> After 2-3 days now I notice that sometimes the multimap is not taken into account at all. So spam arrives which is then rated as 2 and gets through, but the header doesn't show me a test for my multimap blacklist that would have prevented it. I can't yet see a pattern when the multimap is ignored. Could it be that I first check a blacklist and then the whitelist? Is there even an order?
>> What I observe is that the number of checks that I display in the header of the email always varies and has no logic. Sometimes RBL's work and then only my blacklist and the RBL's don't. I don't see any connection yet.
>> But I will continue to monitor this.
>> Trie and error
>> 
>> Thanks
>> Christian
>> 
>> 
>> 
>> German Version
>> 
>> Hallo G.W. Haywood,
>> Danke f?r Deine ausf?hrliche Hilfe. Das hat mir einige Denkanst??e gegeben.
>> 
>> In der Nacht habe ich Rspamd komplett von meinem System entfernt und neu installiert und auch redis gelleert. Ich habe schon von Anfang an die Repository von Rspamd verwendet wie Du beschrieben hast. Ich bin also auf dem neusten Stand.
>> Nach der Neuinstallation habe ich durch rspamadm configtest keine Fehler. Dann habe ich nur das notwendigste eingerichtet. Benutzeraccount f?r das Webfrontend, Redis,ClamAV Verbindung. Und dann beobachtete ich einen Tag was passiert. Ca. 25000 Emails gingen durch Rspamd und ca. 20 waren "false positive". Was eigentlich nicht so schlecht ist. Es waren nur die ?blichen verd?chtigen, die es immer schaffen durchzukommen.
>> 
>> Dann habe ich erst mal meine Whiteliste ?ber multimap eingeh?ngt, da einige wichtige sender werte von +1-3 hatten und knapp an der Spam Grenze waren.
>> Weiter beobachten ..
>> Dann die Blacklist. Die ?blichen verd?chtigen konnte ich jetzt auch rausziehen.
>> Nach jetzt 2-3 Tagen stelle ich doch fest, das manchmal die multimap gar nicht beachtet wird. Es kommt also Spam an die dann aber mit 2 bewertet wird und durchkommt, aber im Header wird mir kein Test auf meine multimap Blacklist angezeigt die es verhindert h?tte. Ich kann noch kein Muster erkennen, wann die multimap ignoriert wird. Kann es dran liegen das ich zuerst eine blacklist pr?fe und dann die whiteliste. Gibt es da ?berhaupt eine Reihenfolge.
>> Was ich beobachte ist das die Anzahl der Pr?fungen, die ich mir im header der Mail anzeigen lasse, immer variiert und keine Logig hat. Manchmal greifen RBL?s und dann aber nur meine blacklist und die RBL?s nicht. Da sehe ich noch keinen Zusammenhang.
>> Ich werde das aber weiter beobachten.
>> Trie and Error
>> 
>> Danke
>> Christian
>> 
>> 
>> Am 13.02.2024 um 13:19 schrieb G.W. Haywood:
>>> Hello again,
>>> On Mon, 12 Feb 2024, christian via Users wrote:
>>>> Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
>>>>> On Mon, 12 Feb 2024, christian via Users wrote:
>>>>> 
>>>>>> ... no evaluation is made based on my multimap ...
>>>>> 
>>>>> Here's something I found useful to try to make sense of things:
>>>>> 
>>>>> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
>>>>> 
>>>>> Just a guess, it might be an entry triggered in a whitelist or for
>>>>> example mid, both of which you probably have enabled.
>>>>> 
>>>>> ... try ... rspamadm configtest
>>>> 
>>>> My rspamadm configtest gives the following:
>>>> 
>>>> rspamadm configtest
>>>> duplicate symbol: SUBJ_ALL_CAPS, skip registering
>>>> unknown type of attribute enabled for regexp module
>>>> cannot find dependency on symbol IS_IN_WHITELIST for symbol FORCE_ACTION_MY_WHITELIST
>>>> cannot find dependency on symbol FPROT_VIRUS for symbol FORCE_ACTION_MY_WHITELIST
>>>> syntax OK
>>>> 
>>>> But unfortunately I can't do anything with the information.
>>>> 
>>>> What's more, I haven't changed anything in the symbols displayed.
>>> With a configuration as complex as that of rspamd, in my view it's
>>> absolutely essential to be very methodical when making any changes.
>>> Do you have a record of all the changes that you made?  Do you have a
>>> copy of your original configuration before you made any changes?  One
>>> of the tools I use a great deal for this sort of work is 'diff'.  It
>>> is, I suppose, really intended for people writing code, but it makes
>>> it easy to compare huge sets of configuration options with changes in
>>> files which hap-hazardly distributed within a large directory tree.
>>> To test the latest installation instructions for rspamd which are
>>> found on the rspamd Website, I followed them.  On a box which runs
>>> Debian 'Bullseye', I ran the script below:
>>> 8<----------------------------------------------------------------------
>>> #!/bin/bash
>>> apt-get install -y lsb-release wget gpg
>>> CODENAME=`lsb_release -c -s`
>>> mkdir -p /etc/apt/keyrings
>>> wget -O- https://rspamd.com/apt-stable/gpg.key | gpg --dearmor | tee /etc/apt/keyrings/rspamd.gpg > /dev/null
>>> echo "deb [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main" | tee /etc/apt/sources.list.d/rspamd.list
>>> echo "deb-src [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main"  | tee -a /etc/apt/sources.list.d/rspamd.list
>>> apt-get update
>>> apt-get --no-install-recommends install rspamd
>>> 8<----------------------------------------------------------------------
>>> It only took a minute or so, and ran to completion with no issues.
>>> The installation script is more or less according to the instructions at
>>> https://rspamd.com/downloads.html
>>> but I removed 'sudo' because I ran the script as root.
>>> Below is what happened with rspamd newly installed using the script:
>>> 8<----------------------------------------------------------------------
>>> # rspamadm --version
>>> Rspamadm 3.8.1
>>> # rspamadm configtest
>>> syntax OK
>>> # ls -l /usr/share/rspamd/
>>> total 264
>>> -rw-r--r--  1 root root 242327 Jan 25 19:10 effective_tld_names.dat
>>> drwxr-xr-x  2 root root   4096 Feb 13 10:45 elastic
>>> drwxr-xr-x  2 root root   4096 Feb 13 10:45 languages
>>> drwxr-xr-x 10 root root   4096 Feb 13 10:45 lualib
>>> drwxr-xr-x  2 root root   4096 Feb 13 10:45 plugins
>>> drwxr-xr-x  4 root root   4096 Feb 13 10:45 rules
>>> drwxr-xr-x  6 root root   4096 Feb 13 10:45 www
>>> # ls -l /etc/rspamd/
>>> total 96
>>> -rw-r--r-- 1 root root 1213 Jan 25 19:10 actions.conf
>>> -rw-r--r-- 1 root root  365 Jan 25 19:10 cgp.inc
>>> -rw-r--r-- 1 root root 1318 Jan 25 19:10 common.conf
>>> -rw-r--r-- 1 root root 7614 Jan 25 19:10 composites.conf
>>> -rw-r--r-- 1 root root 5154 Jan 25 19:10 groups.conf
>>> -rw-r--r-- 1 root root  874 Jan 25 19:10 lang_detection.inc
>>> drwxr-xr-x 2 root root 4096 Jan 25 19:31 local.d
>>> -rw-r--r-- 1 root root 1186 Jan 25 19:10 logging.inc
>>> drwxr-xr-x 2 root root 4096 Feb 13 10:45 maps.d
>>> -rw-r--r-- 1 root root  921 Jan 25 19:10 metrics.conf
>>> -rw-r--r-- 1 root root  703 Jan 25 19:10 modules.conf
>>> drwxr-xr-x 2 root root 4096 Feb 13 10:45 modules.d
>>> -rw-r--r-- 1 root root 2020 Jan 25 19:10 options.inc
>>> drwxr-xr-x 2 root root 4096 Jan 25 19:31 override.d
>>> -rw-r--r-- 1 root root 2787 Jan 25 19:10 rspamd.conf
>>> drwxr-xr-x 2 root root 4096 Feb 13 10:45 scores.d
>>> -rw-r--r-- 1 root root 1799 Jan 25 19:10 settings.conf
>>> -rw-r--r-- 1 root root 2169 Jan 25 19:10 statistic.conf
>>> -rw-r--r-- 1 root root  618 Jan 25 19:10 worker-controller.inc
>>> -rw-r--r-- 1 root root  654 Jan 25 19:10 worker-fuzzy.inc
>>> -rw-r--r-- 1 root root  525 Jan 25 19:10 worker-normal.inc
>>> -rw-r--r-- 1 root root 1363 Jan 25 19:10 worker-proxy.inc
>>> 8<----------------------------------------------------------------------
>>> As you can see the files all have a relatively recent timestamp, which
>>> I find comforting when I do a new installtaion.  Directory timestamps
>>> in this case are when they were created, that's not important here but
>>> if I have trouble finding changes in more or less anything I'll often
>>> look for files/directories in a directory tree with recent timestamps.
>>> It's clear that you've done things with which the rspamd configuration
>>> test is a little unhappy, but it _does_ say it's 'OK' so I think it
>>> will probably be working as designed.  Whether or not it's working as
>>> you intend is another matter.  You could always try the stricter test
>>> according to the 'man' page - this is the result on my new config:
>>> # rspamadm --var=DBDIR=/tmp configtest -c /etc/rspamd/rspamd.conf -s
>>> syntax OK
>>> If it were my system I think I'd want to be sure that all the warnings
>>> were gone before I'd be happy.  Perhaps you could back out the changes
>>> which you've made one-by-one until the output is silenced.  If you do
>>> not have a record of the changes you could start with a fresh config.
>>> Document each and every change you make, but also (1) test it and (2)
>>> dump it to a file each time you change it, with something like
>>> # rspamadm configdump > ~/rspamd.configdump.$(date -Iseconds)
>>> so you always have a record of a sane configuration if you break it.
>>> Unfortunately comparing the outputs of 'configdump' taken at different
>>> times might not be very helpful; the order in which the various parts
>>> are output seems to be non-deterministic.
>>> To address the issue of some mails being passed through without being
>>> processed by some of the modules I think maybe you should investigate
>>> the parts of the configuration which implement whitelisting, but *not*
>>> before you're happy that you know exactly what changes you have made
>>> to the configuration and exactly what you expect these changes to do.
>>> If you still can't figure out why you aren't getting the results you
>>> expect, you can post to this list the changes which you have made (or
>>> as I said earlier the full configuration dump).
>>> If you have an example mail which was blocked by your configuration,
>>> (and should have been) and one which you feel is similar but was not
>>> (and should have been) perhaps you can put them somewhere where we
>>> can see them and try to do some guesswork.
>>>> If I understand correctly, Rspamd's basic settings are fetched from
>>>> /usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d.
>>>> Some like multimap have no basic settings and are created themselves
>>>> and read by rspamd.
>>> Depending on how you look at things, fortunately or unfortunately the
>>> paths for the configuration directories are themselves configurable.
>>> That means the exact locations can be different in different systems
>>> and the single biggest factors which determine the locations are the
>>> way you installed rspamd and from where you downloaded the package.
>>> If you installed as per the instruction on the rspamd Website your
>>> understanding is correct.  I do not know what might happen if you
>>> installed using packages from your Linux distribution, but if for
>>> example you used the Debian package with just
>>> apt-get install rspamd
>>> there's a good chance that things will be four or five years out of
>>> date and horribly broken.
>>>> As a non-English speaker, I find it quite difficult to use the
>>>> docs. Excuse me.
>>> Your English is good but the documentation on the rspamd Website is I
>>> fear not ideal for a new user.  That's why I pointed to an alternative
>>> which, although it is somewhat out of date, tries a bit harder to make
>>> the configuration make sense to someone new.  The array of options in
>>> the rspamd configuration is immense and it's too easy to find yourself
>>> lost in the woods.
>> -- 
>> Users mailing list
>> Users at lists.rspamd.com
>> https://lists.rspamd.com/mailman/listinfo/users
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4335 bytes
Desc: not available
URL: <https://lists.rspamd.com/pipermail/users/attachments/20240215/3864012e/attachment.bin>

From rspamd at jubileegroup.co.uk  Thu Feb 15 17:23:24 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Thu, 15 Feb 2024 17:23:24 +0000 (GMT)
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
Message-ID: <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>

Hi there,

On Thu, 15 Feb 2024, christian via Users wrote:

> ...
> After reinstalling I have no errors through rspamadm configtest.

:)

> Then I only set up the bare essentials. ... Around 25,000 emails ...
> 20 were ?false positives?. Which isn't actually that bad. ...

:)

> ... sometimes the multimap is not taken into account ...
> Could it be that I first check a blacklist and then the whitelist?
> Is there even an order?

Does this help?

https://rspamd.com/doc/faq.html#what-are-filters-pre-filters-and-post-filters

-- 

73,
Ged.

From t.hendricks at interpool.de  Fri Feb 16 09:56:05 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Fri, 16 Feb 2024 10:56:05 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
Message-ID: <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>

Hi Ged,

if I have 

rspamadm configdump
--
    SENDER_FROM_WHITELIST {
        action = "accept";
        type = "from";
        map = "//etc/rspamd/local.d/from_whitelist.map";
        prefilter = true;
    }
--

shouldn?t this suffice for the from_whitelist.map to act on an incoming mail before any other module hits?

Thanks for this awesome piece of software!

Tino

> Am 15.02.2024 um 18:23 schrieb G.W. Haywood <rspamd at jubileegroup.co.uk>:
> 
> Hi there,
> 
> On Thu, 15 Feb 2024, christian via Users wrote:
> 
>> ...
>> After reinstalling I have no errors through rspamadm configtest.
> 
> :)
> 
>> Then I only set up the bare essentials. ... Around 25,000 emails ...
>> 20 were ?false positives?. Which isn't actually that bad. ...
> 
> :)
> 
>> ... sometimes the multimap is not taken into account ...
>> Could it be that I first check a blacklist and then the whitelist?
>> Is there even an order?
> 
> Does this help?
> 
> https://rspamd.com/doc/faq.html#what-are-filters-pre-filters-and-post-filters
> 
> -- 
> 
> 73,
> Ged.
> -- 
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users


From rspamd at jubileegroup.co.uk  Fri Feb 16 14:06:32 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Fri, 16 Feb 2024 14:06:32 +0000 (GMT)
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
 <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
Message-ID: <33f9fc49-a71a-8c2d-32ef-328c578da929@jubileegroup.co.uk>

Hi there,

On Fri, 16 Feb 2024, Tino Hendricks via Users wrote:

> if I have
>
> rspamadm configdump
> --
>    SENDER_FROM_WHITELIST {
>        action = "accept";
>        type = "from";
>        map = "//etc/rspamd/local.d/from_whitelist.map";
>        prefilter = true;
>    }
> --
>
> shouldn?t this suffice for the from_whitelist.map to act on an incoming mail before any other module hits?

It's my inderstanding that you should use scores, not actions, to
control behaviour and that the use of 'prefilter' without a full
understanding of the way rspamd configuration works is a bad idea.
I guess if you have to ask the question then it must be a bad idea.

I'm only parroting this advice:

https://lists.rspamd.com/pipermail/users/2021-March/001990.html

and I'm sure someone will correct me if I'm wrong.

> Thanks for this awesome piece of software!

Don't blame me, I only work here. ;)

-- 

73,
Ged.

From t.hendricks at interpool.de  Fri Feb 16 14:30:17 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Fri, 16 Feb 2024 15:30:17 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <33f9fc49-a71a-8c2d-32ef-328c578da929@jubileegroup.co.uk>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
 <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
 <33f9fc49-a71a-8c2d-32ef-328c578da929@jubileegroup.co.uk>
Message-ID: <DA1E35C3-A482-4F86-9C5F-3517BBB7BDD0@interpool.de>

Hi Ged,

thanks again for you valuable time, I really appreciate it!

I took the action-thing from
https://rspamd.com/doc/modules/multimap.html#pre-filter-maps

OK, I?ll go for scores now, thanks a lot!

Tino

> Am 16.02.2024 um 15:06 schrieb G.W. Haywood <rspamd at jubileegroup.co.uk>:
> 
> Hi there,
> 
> On Fri, 16 Feb 2024, Tino Hendricks via Users wrote:
> 
>> if I have
>> 
>> rspamadm configdump
>> --
>>   SENDER_FROM_WHITELIST {
>>       action = "accept";
>>       type = "from";
>>       map = "//etc/rspamd/local.d/from_whitelist.map";
>>       prefilter = true;
>>   }
>> --
>> 
>> shouldn?t this suffice for the from_whitelist.map to act on an incoming mail before any other module hits?
> 
> It's my inderstanding that you should use scores, not actions, to
> control behaviour and that the use of 'prefilter' without a full
> understanding of the way rspamd configuration works is a bad idea.
> I guess if you have to ask the question then it must be a bad idea.
> 
> I'm only parroting this advice:
> 
> https://lists.rspamd.com/pipermail/users/2021-March/001990.html
> 
> and I'm sure someone will correct me if I'm wrong.
> 
>> Thanks for this awesome piece of software!
> 
> Don't blame me, I only work here. ;)
> 
> -- 
> 
> 73,
> Ged.
> -- 
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users


From usenet at schani.com  Fri Feb 16 15:26:50 2024
From: usenet at schani.com (christian)
Date: Fri, 16 Feb 2024 16:26:50 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <33f9fc49-a71a-8c2d-32ef-328c578da929@jubileegroup.co.uk>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
 <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
 <33f9fc49-a71a-8c2d-32ef-328c578da929@jubileegroup.co.uk>
Message-ID: <077a16e9-f1be-4dac-b51a-ccf6faac891a@schani.com>

I'm not entirely sure about that either.
The docs say - "If a map matches, no filters will be processed for a 
message."

https://rspamd.com/doc/modules/multimap.html#pre-filter-maps

That would make the scores superfluous since no further testing takes 
place anyway. So the action seems to be important in connection with 
prefilter.
I would just
map = "//etc/rspamd/local.d/from_whitelist.map"; with
map = "/etc/rspamd/local.d/from_whitelist.map"; substitute.



Am 16.02.2024 um 15:06 schrieb G.W. Haywood:

> It's my inderstanding that you should use scores, not actions, to
> control behaviour and that the use of 'prefilter' without a full
> understanding of the way rspamd configuration works is a bad idea.
> I guess if you have to ask the question then it must be a bad idea.
> 
> I'm only parroting this advice:
> 
> https://lists.rspamd.com/pipermail/users/2021-March/001990.html

Pretty confusing. Nothing is quite clear about the matter.


> and I'm sure someone will correct me if I'm wrong.
> 
>> Thanks for this awesome piece of software!
> 
> Don't blame me, I only work here. ;)
> 

trial and error

Christian

From rspamd at jubileegroup.co.uk  Fri Feb 16 18:14:57 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Fri, 16 Feb 2024 18:14:57 +0000 (GMT)
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <077a16e9-f1be-4dac-b51a-ccf6faac891a@schani.com>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
 <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
 <33f9fc49-a71a-8c2d-32ef-328c578da929@jubileegroup.co.uk>
 <077a16e9-f1be-4dac-b51a-ccf6faac891a@schani.com>
Message-ID: <a7d789e0-daeb-40e4-646-4c93a3644652@jubileegroup.co.uk>

Hello again,

On Fri, 16 Feb 2024, christian via Users wrote:
> Am 16.02.2024 um 15:06 schrieb G.W. Haywood:
>
>> ...
>> I'm only parroting this advice:
>> 
>> https://lists.rspamd.com/pipermail/users/2021-March/001990.html
>
> Pretty confusing. Nothing is quite clear about the matter.

The advice was given by the author of rspamd, so I'm thinking that we
should pay more attention to it than if it came from elsewhere.

-- 

73,
Ged.


From list+rspamd at gcore.biz  Fri Feb 16 22:25:27 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Fri, 16 Feb 2024 23:25:27 +0100
Subject: [Rspamd-Users] Things I don't understand yet
In-Reply-To: <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
References: <6a07d8bc-1e90-4751-b0e1-ec8af9d891e0@schani.com>
 <fd1cc128-d843-f847-5f2d-233dbe4e4fa@jubileegroup.co.uk>
 <d4af0743-4ac8-4a91-9642-877625d2043f@schani.com>
 <e8f9f386-322-43c7-9692-8b767a51b05a@jubileegroup.co.uk>
 <81acf375-9e58-4e84-ab75-cb738c87a56a@schani.com>
 <f76d28b6-81c2-e52d-84fd-90e8f6522811@jubileegroup.co.uk>
 <A1667E4B-B269-4A29-A331-4ACC9EE276D8@interpool.de>
Message-ID: <97CEF685-F4BE-48CB-ACBD-CCC758F7D001@gcore.biz>

>    SENDER_FROM_WHITELIST {
>        action = "accept";
>        type = "from";
>        map = "//etc/rspamd/local.d/from_whitelist.map";
>        prefilter = true;
>    }

There also is a module for whitelisting, see:
https://rspamd.com/doc/modules/whitelist.html

A multimap with "action = accept" stops processing, which is usually
not what you want. I'm using it for special addresses that do not
want any filtering.

A better approach for you might be to just set symbols upon multimap
matches. To avoid bayes learning, set low scores. Then you can combine
symbols with force_actions and act accordingly, see:
https://rspamd.com/doc/modules/force_actions.html

Example from that page:

  MY_WHITELIST {
    # This is the action we want to force
    action = "no action";
    # If the following combination of symbols is present:
    expression = "IS_IN_WHITELIST & !CLAM_VIRUS & !FPROT_VIRUS";
  }

It implements a kind of whitelist for non-virus mails only.

This way you can build very flexible combinations for mails that
match multiple maps/symbols.

Best regards,
Gerald


From j.echter at echter-kuechen-elektro.de  Wed Feb 21 08:18:20 2024
From: j.echter at echter-kuechen-elektro.de (=?utf-8?q?J=C3=BCrgen_Echter?=)
Date: Wed, 21 Feb 2024 09:18:20 +0100
Subject: [Rspamd-Users] =?utf-8?q?Since_update_to_rspamd_3=2E8=2E2_i_get_?=
 =?utf-8?b?Q2Fubm90IHJlY2VpdmUgaGlzdG9yeTo/PT0/dXRmLTg/cT8geyJlcnJvciI6?=
 =?utf-8?q?=3F=3D=3D=3Futf-8=3Fq=3F_=22WRONGTYPE_Operation_against_a_key_h?=
 =?utf-8?q?olding_the_wrong_kind_of_value=22=7D?=
Message-ID: <21da8f-65d5b200-3-161091c0@149395606>


Hi,

since i upgrade to rspamd 3.8.2 i get Cannot receive history: {"error": "WRONGTYPE Operation against a key holding the wrong kind of value"}

Is there an easy way to fix that?

Greeting,

Juergen
?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5655 bytes
Desc: not available
URL: <https://lists.rspamd.com/pipermail/users/attachments/20240221/47798a0d/attachment.bin>

From albrecht.backhaus at gmail.com  Wed Feb 21 08:23:41 2024
From: albrecht.backhaus at gmail.com (Albrecht Backhaus)
Date: Wed, 21 Feb 2024 09:23:41 +0100
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {"error":=?utf-8?q? "WRONGTYPE Operation against a key
 holding the wrong kind of value"}
In-Reply-To: <21da8f-65d5b200-3-161091c0@149395606>
References: <21da8f-65d5b200-3-161091c0@149395606>
Message-ID: <b0501399-bc18-4d7d-8087-bc942ed3e489@gmail.com>

------------------------------------------------------------------------
*Von:/From:* J?rgen Echter via Users <users at lists.rspamd.com>
*Gesendet:/Sent:* Mittwoch, 21.02.2024 - 09:18
*An:/To:* User questions <users at lists.rspamd.com>
*Kopie:/CC:* J?rgen Echter <j.echter at echter-kuechen-elektro.de>
*Betreff:/Subject:* [Rspamd-Users] Since update to rspamd 3.8.2 i get 
Cannot receive history:?= {"error":=?utf-8?q? "WRONGTYPE Operation 
against a key holding the wrong kind of value"}
> Hi,
>
> since i upgrade to rspamd 3.8.2 i get Cannot receive history: {"error": "WRONGTYPE Operation against a key holding the wrong kind of value"}
>
> Is there an easy way to fix that?
>
> Greeting,
>
> Juergen
>   
Maybe you need to clear your browser cache.

From j.echter at echter-kuechen-elektro.de  Wed Feb 21 08:31:52 2024
From: j.echter at echter-kuechen-elektro.de (=?utf-8?q?J=C3=BCrgen_Echter?=)
Date: Wed, 21 Feb 2024 09:31:52 +0100
Subject: [Rspamd-Users] 
 =?utf-8?q?Since_update_to_rspamd_3=2E8=2E2_i_get_?=
 =?utf-8?b?Q2Fubm90IHJlY2VpdmUgaGlzdG9yeTo/PSB7ImVycm9yIjo/PT0/dXRmLTg/?=
 =?utf-8?q?q=3F_=22WRONGTYPE_Operation_against_a_key_holding_the_wrong_kin?=
 =?utf-8?q?d_of_value=22?=
In-Reply-To: <b0501399-bc18-4d7d-8087-bc942ed3e489@gmail.com>
References: <21da8f-65d5b200-3-161091c0@149395606>
 <b0501399-bc18-4d7d-8087-bc942ed3e489@gmail.com>
Message-ID: <21ddb1-65d5b500-1-69c03400@157163534>


Am Mittwoch, Februar 21, 2024 09:23 CET, schrieb Albrecht Backhaus <albrecht.backhaus at gmail.com>:
?------------------------------------------------------------------------
*Von:/From:* J?rgen Echter via Users <users at lists.rspamd.com>
*Gesendet:/Sent:* Mittwoch, 21.02.2024 - 09:18
*An:/To:* User questions <users at lists.rspamd.com>
*Kopie:/CC:* J?rgen Echter <j.echter at echter-kuechen-elektro.de>
*Betreff:/Subject:* [Rspamd-Users] Since update to rspamd 3.8.2 i get
Cannot receive history:?= {"error":=?utf-8?q? "WRONGTYPE Operation
against a key holding the wrong kind of value"}
> Hi,
>
> since i upgrade to rspamd 3.8.2 i get Cannot receive history: {"error": "WRONGTYPE Operation against a key holding the wrong kind of value"}
>
> Is there an easy way to fix that?
>
> Greeting,
>
> Juergen
>
Maybe you need to clear your browser cache.
--
Users mailing list
Users at lists.rspamd.com
https://lists.rspamd.com/mailman/listinfo/users

That doesn't work either. I also tried opening the webui in 2 other browser with the same result.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5655 bytes
Desc: not available
URL: <https://lists.rspamd.com/pipermail/users/attachments/20240221/10f65082/attachment.bin>

From r.seffner at seffner-schlesier.de  Wed Feb 21 14:56:44 2024
From: r.seffner at seffner-schlesier.de (Ronny Seffner)
Date: Wed, 21 Feb 2024 14:56:44 +0000
Subject: [Rspamd-Users] *** SPAM-Verdacht [8.31] *** Since update to
 rspamd 3.8.2 i get Cannot receive history:?= {\"error\":=?utf-8?q?
 \"WRONGTYPE Operation against a key holding the wrong kind of value\"}
In-Reply-To: 21da8f-65d5b200-3-161091c0@149395606
References: 21da8f-65d5b200-3-161091c0@149395606
Message-ID: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>

> since i upgrade to rspamd 3.8.2 i get Cannot receive history: {"error":
> "WRONGTYPE Operation against a key holding the wrong kind of value"}
>
Same here (Debian12).
Stop/start of redis and/or rspamd does not help.
Ohter browser or clearing cache does not help.
Then I tried this without success (as it seems - just for me - maybe related) : https://github.com/rspamd/rspamd/pull/4826/files

So I opened a bug report : https://github.com/rspamd/rspamd/issues/4828


Ronny Seffner

From katharina.knuth at icloud.com  Thu Feb 22 09:24:48 2024
From: katharina.knuth at icloud.com (Katharina Knuth)
Date: Thu, 22 Feb 2024 10:24:48 +0100
Subject: [Rspamd-Users] *** SPAM-Verdacht [8.31] *** Since update to
 rspamd 3.8.2 i get Cannot receive history:?= {\"error\":=?utf-8?q?
 \"WRONGTYPE Operation against a key holding the wrong kind of value\"}
In-Reply-To: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
Message-ID: <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>

Am 21.02.24 um 15:56 schrieb Ronny Seffner via Users:
>> since i upgrade to rspamd 3.8.2 i get Cannot receive history: {"error":
>> "WRONGTYPE Operation against a key holding the wrong kind of value"}
>>
> Same here (Debian12).
> Stop/start of redis and/or rspamd does not help.
> Ohter browser or clearing cache does not help.
> Then I tried this without success (as it seems - just for me - maybe related) : https://github.com/rspamd/rspamd/pull/4826/files
> 
> So I opened a bug report : https://github.com/rspamd/rspamd/issues/4828
> 
> 
> Ronny Seffner

Since 9:40 working that again. Also Debian 12

-- 
freundliche Gr??e,
Sincerely yours,

Katharina Knuth


From r.seffner at seffner-schlesier.de  Thu Feb 22 17:38:23 2024
From: r.seffner at seffner-schlesier.de (Ronny Seffner)
Date: Thu, 22 Feb 2024 17:38:23 +0000
Subject: [Rspamd-Users] *** SPAM-Verdacht [9.41] *** Re: ***
 SPAM-Verdacht [8.31] *** Since update to rspamd 3.8.2 i get Cannot receive
 history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation against a key
 holding the wrong kind of value\\\"}
In-Reply-To: <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
Message-ID: <218346af1c7e4d598d7e98fdb0cc2eb1@seffner-schlesier.de>

> Since 9:40 working that again. Also Debian 12
>
Not here ;-(

What chaned at this time at your setup - maybe there is any cache involved I don't expect.


Mit freundlichen Gr??en

Ronny Seffner

From katharina.knuth at icloud.com  Thu Feb 22 17:44:56 2024
From: katharina.knuth at icloud.com (Katharina Knuth)
Date: Thu, 22 Feb 2024 18:44:56 +0100
Subject: [Rspamd-Users] *** SPAM-Verdacht [9.41] *** Re: ***
 SPAM-Verdacht [8.31] *** Since update to rspamd 3.8.2 i get Cannot receive
 history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation against a key
 holding the wrong kind of value\\\"}
In-Reply-To: <218346af1c7e4d598d7e98fdb0cc2eb1@seffner-schlesier.de>
References: <218346af1c7e4d598d7e98fdb0cc2eb1@seffner-schlesier.de>
Message-ID: <81AA8D23-383A-45B5-99F9-C825578E3C82@icloud.com>

I don?t know, I?ve doing nothing 

liebe Gr??e,
kind regards,
Katharina Knuth
(Mobile)

> Am 22.02.2024 um 18:38 schrieb Ronny Seffner <r.seffner at seffner-schlesier.de>:
> 
> ?
>> 
>> Since 9:40 working that again. Also Debian 12
>> 
> Not here ;-(
> 
> What chaned at this time at your setup - maybe there is any cache involved I don't expect.
> 
> 
> Mit freundlichen Gr??en
> 
> Ronny Seffner

From r.seffner at seffner-schlesier.de  Fri Feb 23 09:30:03 2024
From: r.seffner at seffner-schlesier.de (Ronny Seffner)
Date: Fri, 23 Feb 2024 09:30:03 +0000
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation
 against a key holding the wrong kind of value\\\"}
In-Reply-To: <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
Message-ID: <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>

As in bug report at github found, following works for me:

"it works for me if I set the key_prefix in local.d/history_redis.conf to "rs_history{{HOSTNAME}}{{COMPRESS}}"; instead of "rs_history";."


Ronny Seffner

From r.seffner at seffner-schlesier.de  Fri Feb 23 09:40:00 2024
From: r.seffner at seffner-schlesier.de (Ronny Seffner)
Date: Fri, 23 Feb 2024 09:40:00 +0000
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation
 against a key holding the wrong kind of value\\\"}
In-Reply-To: <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
Message-ID: <2ccc2b542bf743e79f4c19f871f45d6e@seffner-schlesier.de>

Another solution is

echo 'DEL rs_history' | redis-cli



Ronny Seffner

From usenet at schani.com  Sun Feb 25 13:36:36 2024
From: usenet at schani.com (christian)
Date: Sun, 25 Feb 2024 14:36:36 +0100
Subject: [Rspamd-Users] Incorrect filtering in multimap?
Message-ID: <14d89e29-b05a-4f99-a604-bdd1c05305eb@schani.com>

Hello,
I've actually had good experiences with Rspamd for weeks now. Of course, 
I observe the results of the filtering and notice that spam emails slip 
through every now and then. But for strange reasons.
An email just came through from info at beepost.de with a picture book 
spam. I have this domain in a blacklist via multimap. But the email 
comes through with the note

WHITELIST_SENDER_DOMAIN (-20) [beepost.de]
R_DKIM_ALLOW (-0.2) [beepost.de:s=dfxd2023,beepost.de:s=aventura-1k-a]
R_SPF_ALLOW (-0.2) [+ip4:185.212.196.112/31]
MAILLIST (-0.1125) [generic]
MIME_GOOD (-0.1) [multipart/related,multipart/alternative,text/plain]
REPLYTO_EQ_FROM (0)
FROM_HAS_DN (0)
FROM_NEQ_ENVFROM (0) [info at beepost.de,rtpath at beepost.de]
TAGGED_FROM (0) [0224b716db65822044660000198b4515]
ASN (0) [asn:8426, ipnet:185.212.196.0/22, country:GB]
MIME_TRACE (0) [0:+,1:+,2:+,3:~]
HAS_REPLYTO (0) [info at beepost.de]
DKIM_TRACE (0) [beepost.de:+]

Of course I checked and the domain is not included in my whitelist. I 
checked all maps. The domain is only in the blacklist. There is also no 
Bayes test for the email, and no SPAMD check, which should also 
recognize it as spam.
Why does something like this happen? configtest returns OK

Do you have an idea what it could be?
Thanks
Christian

From rspamd at vlh.dk  Sun Feb 25 13:50:43 2024
From: rspamd at vlh.dk (rspamd at vlh.dk)
Date: Sun, 25 Feb 2024 14:50:43 +0100
Subject: [Rspamd-Users] Incorrect filtering in multimap?
In-Reply-To: <14d89e29-b05a-4f99-a604-bdd1c05305eb@schani.com>
References: <14d89e29-b05a-4f99-a604-bdd1c05305eb@schani.com>
Message-ID: <000001da67f1$a0614740$e123d5c0$@vlh.dk>

Can you show relevant part(s) of your multimap.conf - could you have
referenced wrong file in your setup? (ie. swapped black and whitelists)

-Kim Sindalsen

> -----Original Message-----
> From: Users <users-bounces at lists.rspamd.com> On Behalf Of christian via
> Users
> Sent: 25. februar 2024 14:37
> To: User questions <users at lists.rspamd.com>
> Cc: christian <usenet at schani.com>
> Subject: [Rspamd-Users] Incorrect filtering in multimap?
> 
> Hello,
> I've actually had good experiences with Rspamd for weeks now. Of course, I
> observe the results of the filtering and notice that spam emails slip
through
> every now and then. But for strange reasons.
> An email just came through from info at beepost.de with a picture book spam.
> I have this domain in a blacklist via multimap. But the email comes
through
> with the note
> 
> WHITELIST_SENDER_DOMAIN (-20) [beepost.de] R_DKIM_ALLOW (-0.2)
> [beepost.de:s=dfxd2023,beepost.de:s=aventura-1k-a]
> R_SPF_ALLOW (-0.2) [+ip4:185.212.196.112/31] MAILLIST (-0.1125)
> [generic] MIME_GOOD (-0.1)
> [multipart/related,multipart/alternative,text/plain]
> REPLYTO_EQ_FROM (0)
> FROM_HAS_DN (0)
> FROM_NEQ_ENVFROM (0) [info at beepost.de,rtpath at beepost.de]
> TAGGED_FROM (0) [0224b716db65822044660000198b4515]
> ASN (0) [asn:8426, ipnet:185.212.196.0/22, country:GB] MIME_TRACE (0)
> [0:+,1:+,2:+,3:~] HAS_REPLYTO (0) [info at beepost.de] DKIM_TRACE (0)
> [beepost.de:+]
> 
> Of course I checked and the domain is not included in my whitelist. I
checked
> all maps. The domain is only in the blacklist. There is also no Bayes test
for the
> email, and no SPAMD check, which should also recognize it as spam.
> Why does something like this happen? configtest returns OK
> 
> Do you have an idea what it could be?
> Thanks
> Christian
> --
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users


From a.wass at glas-gasperlmair.at  Mon Feb 26 10:24:37 2024
From: a.wass at glas-gasperlmair.at (Andreas Wass - Glas Gasperlmair)
Date: Mon, 26 Feb 2024 11:24:37 +0100
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation
 against a key holding the wrong kind of value\\\"}
In-Reply-To: <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
Message-ID: <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>

Same Problem here.

best regards Andi

Am 23.02.2024 um 10:30 schrieb Ronny Seffner via Users:
> As in bug report at github found, following works for me:
Cannot find this bug in github rspamd issues
Any link to this?
>
> "it works for me if I set the key_prefix in local.d/history_redis.conf to "rs_history{{HOSTNAME}}{{COMPRESS}}"; instead of "rs_history";."
is this an official solution?
>
>
> Ronny Seffner


From r.seffner at seffner-schlesier.de  Mon Feb 26 10:51:34 2024
From: r.seffner at seffner-schlesier.de (Ronny Seffner)
Date: Mon, 26 Feb 2024 10:51:34 +0000
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {\\\\\\\"error\\\\\\\":=?utf-8?q? \\\\\\\"WRONGTYPE
 Operation against a key holding the wrong kind of value\\\\\\\"}
In-Reply-To: <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
 <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
Message-ID: <902bf55b34dd4b99a656353d2bbae13d@seffner-schlesier.de>

> is this an official solution?
>
Official ist to clear redis as maintainer/developer wrote at github.


Ronny Seffner

From rspamd at vlh.dk  Mon Feb 26 10:52:03 2024
From: rspamd at vlh.dk (Kim Sindalsen)
Date: Mon, 26 Feb 2024 11:52:03 +0100
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation
 against a key holding the wrong kind of value\\\"}
In-Reply-To: <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
 <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
Message-ID: <ff0b0cfee2c6bad799cd69fc89ecf1f7@vlh.dk>

Hi,

Initally I thought I didn't have any bug after upgrading - didn't get 
any error on history-tab, but found out it had simply stopped updating.

I used the solution here - 
https://github.com/rspamd/rspamd/issues/4828#issuecomment-1960995820
(echo 'DEL rs_history' | redis-cli)

from Vsevolod Stakhov - worked perfectly fine for me.

-Kim Sindalsen


On 2024-02-26 11:24, Andreas Wass - Glas Gasperlmair wrote:
> Same Problem here.
> 
> best regards Andi
> 
> Am 23.02.2024 um 10:30 schrieb Ronny Seffner via Users:
>> As in bug report at github found, following works for me:
> Cannot find this bug in github rspamd issues
> Any link to this?
>> 
>> "it works for me if I set the key_prefix in local.d/history_redis.conf 
>> to "rs_history{{HOSTNAME}}{{COMPRESS}}"; instead of "rs_history";."
> is this an official solution?
>> 
>> 
>> Ronny Seffner

From vsevolod at rspamd.com  Mon Feb 26 10:55:07 2024
From: vsevolod at rspamd.com (Vsevolod Stakhov)
Date: Mon, 26 Feb 2024 10:55:07 +0000
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:
In-Reply-To: <902bf55b34dd4b99a656353d2bbae13d@seffner-schlesier.de>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
 <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
 <902bf55b34dd4b99a656353d2bbae13d@seffner-schlesier.de>
Message-ID: <a59137ee-5141-3de3-123a-3e49a65bcbbe@rspamd.com>

On 26/02/2024 10:51, Ronny Seffner via Users wrote:
>> is this an official solution?
>>
> Official ist to clear redis as maintainer/developer wrote at github.
> 
> 
> Ronny Seffner

The proper way is not to redefine defaults that you don't understand. 
But nobody listens for my voice anyway.

From a.wass at glas-gasperlmair.at  Mon Feb 26 11:59:29 2024
From: a.wass at glas-gasperlmair.at (Andreas Wass - Glas Gasperlmair)
Date: Mon, 26 Feb 2024 12:59:29 +0100
Subject: [Rspamd-Users] Since update to rspamd 3.8.2 i get Cannot
 receive history:?= {\\\"error\\\":=?utf-8?q? \\\"WRONGTYPE Operation
 against a key holding the wrong kind of value\\\"}
In-Reply-To: <ff0b0cfee2c6bad799cd69fc89ecf1f7@vlh.dk>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
 <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
 <ff0b0cfee2c6bad799cd69fc89ecf1f7@vlh.dk>
Message-ID: <d50f5e5b-4e09-4483-b575-afa02550c070@glas-gasperlmair.at>

Thank you Kim, this link is really helpful!

I did'nt find it, bcause it's actually closed and i was searching only 
for open issues.

greetings, Andi

Am 26.02.2024 um 11:52 schrieb Kim Sindalsen via Users:
> Hi,
>
> Initally I thought I didn't have any bug after upgrading - didn't get 
> any error on history-tab, but found out it had simply stopped updating.
>
> I used the solution here - 
> https://github.com/rspamd/rspamd/issues/4828#issuecomment-1960995820
> (echo 'DEL rs_history' | redis-cli)
>
> from Vsevolod Stakhov - worked perfectly fine for me.
>
> -Kim Sindalsen
>
>
> On 2024-02-26 11:24, Andreas Wass - Glas Gasperlmair wrote:
>> Same Problem here.
>>
>> best regards Andi
>>
>> Am 23.02.2024 um 10:30 schrieb Ronny Seffner via Users:
>>> As in bug report at github found, following works for me:
>> Cannot find this bug in github rspamd issues
>> Any link to this?
>>>
>>> "it works for me if I set the key_prefix in 
>>> local.d/history_redis.conf to "rs_history{{HOSTNAME}}{{COMPRESS}}"; 
>>> instead of "rs_history";."
>> is this an official solution?
>>>
>>>
>>> Ronny Seffner


From j.echter at echter-kuechen-elektro.de  Tue Feb 27 11:27:45 2024
From: j.echter at echter-kuechen-elektro.de (=?utf-8?q?J=C3=BCrgen_Echter?=)
Date: Tue, 27 Feb 2024 12:27:45 +0100
Subject: [Rspamd-Users] 
 =?utf-8?q?Since_update_to_rspamd_3=2E8=2E2_i_get_?=
 =?utf-8?q?Cannot_receive_history=3A?=
In-Reply-To: <a59137ee-5141-3de3-123a-3e49a65bcbbe@rspamd.com>
References: <2498fd5838ba420095456d497636016f@seffner-schlesier.de>
 <24c1486e-7393-4207-9e8e-191e3fa8086d@icloud.com>
 <40c18cb5139240e1891350c265b47201@seffner-schlesier.de>
 <bd3664ab-e06f-4018-9cee-e1da3523797c@glas-gasperlmair.at>
 <902bf55b34dd4b99a656353d2bbae13d@seffner-schlesier.de>
 <a59137ee-5141-3de3-123a-3e49a65bcbbe@rspamd.com>
Message-ID: <df755-65ddc700-3-4614df80@71263751>


after i found the solution on github issues i realized what my issue was.

I did redefine a default i shouldn't touch in my local.d.

I fully understand your pain.

Am Montag, Februar 26, 2024 11:55 CET, schrieb Vsevolod Stakhov <vsevolod at rspamd.com>:
?On 26/02/2024 10:51, Ronny Seffner via Users wrote:
>> is this an official solution?
>>
> Official ist to clear redis as maintainer/developer wrote at github.
>
>
> Ronny Seffner

The proper way is not to redefine defaults that you don't understand.
But nobody listens for my voice anyway.
--
Users mailing list
Users at lists.rspamd.com
https://lists.rspamd.com/mailman/listinfo/users
?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5655 bytes
Desc: not available
URL: <https://lists.rspamd.com/pipermail/users/attachments/20240227/1dc4c887/attachment.bin>

From usenet at schani.com  Wed Feb 28 11:52:03 2024
From: usenet at schani.com (christian)
Date: Wed, 28 Feb 2024 12:52:03 +0100
Subject: [Rspamd-Users] Multimap and syntax...
Message-ID: <03bdf4ff-f8c1-4706-9df1-862397d977d3@schani.com>

Hello,
I still have the problem that my multimaps don't work reliably. 
Sometimes they are not used at all, even though it is obvious that they 
should work.
Some of the syntax is not clear to me either. Here's an example:

#Domain Whitelisting
WHITELIST_SENDER_DOMAIN {
type = "from";
filter = "email:domain:tld";
symbol = "WHITELIST_SENDER_DOMAIN";
map = "/etc/rspamd/maps.d/whitelist.sender.domain.map";
score = -20.0;
};

I use the filter email:domain:tld which according to Docs Somebody 
"<user at foo.example.com> -> example.com " only returns the domain. So I 
enter e.g. aok.de in my map. But what about: canford.co.uk? Is co.uk 
then considered a TLD or as a domain and tld.
Should I do this with regex or not? I have many domains like this that I 
need to whitelist, but I'm not sure how to do it:
cmp.dotmailer.co.uk
de-news.adidas.com
de.email.lightinthebox.com
delivery.shop-apotheke.com
e.change.org
e.familysearch.org
eudora.co.nz
eue.week.industr.com
reply.ebay.de
reply2.ebay.de

The problem I have is that some entries just don't work and I don't know 
why that could be. In my current white domain list I have around 2000 
entries. Could it be that there are too many?

It's exactly the same with the blacklist. Some entries simply don't work.
I have the impression that you are dependent on other tests from RSPAMD, 
but I cannot prove or understand this.
I checked my map files several times for the correct line breaks, 
duplicate entries and special characters. I can't figure it out.

Do you have any other suggestions?
Christian
rpsmad 3.8.4
rspamadm configtest = OK

From list+rspamd at gcore.biz  Wed Feb 28 14:15:39 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Wed, 28 Feb 2024 15:15:39 +0100
Subject: [Rspamd-Users] Multimap and syntax...
In-Reply-To: <03bdf4ff-f8c1-4706-9df1-862397d977d3@schani.com>
References: <03bdf4ff-f8c1-4706-9df1-862397d977d3@schani.com>
Message-ID: <5FDD5A1F-DF46-4E34-8761-7F10B0C96C5E@gcore.biz>

> I use the filter email:domain:tld which according to Docs Somebody "<user at foo.example.com> -> example.com " only returns the domain. So I enter e.g. aok.de in my map. But what about: canford.co.uk? Is co.uk then considered a TLD or as a domain and tld.

Rspamd includes the public suffix list (see https://publicsuffix.org/list/).
https://github.com/rspamd/rspamd/blob/master/contrib/publicsuffix/effective_tld_names.dat

> Should I do this with regex or not?

With hyperscan enabled you can use lots of regexes without performance penalty.
On the other hand you need to be familiar with regular expressions and be excact.
Given the problems you currently have I don't recommend it because it's harder to debug.

> I have many domains like this that I need to whitelist, but I'm not sure how to do it:
> cmp.dotmailer.co.uk

You've copied the example "email:domain:tld" which converts user at foo.example.com to example.com.
So user at cmp.dotmail.co.uk will be converted to dotmail.co.uk, which is not in your list and therefore does not match.

> de-news.adidas.com

Use "email:domain" if you want to match that, not email:domain:tld or remove de-news.adidas.com and add adidas.com instead.

> The problem I have is that some entries just don't work

Try to be more precise when reading the documentation.

> and I don't know why that could be. In my current white domain list I have around 2000 entries. Could it be that there are too many?

Generally speaking, no.

Just a hint: if you add e.g. adidas.com to your whitelist, any spammer that sends with @adidas.com is probably whitelisted due to score -20.
I'd rather train rspamd to filter spam and use those maps to assist learning. Otherwise a spammail with an added score of -20 will probably be learned as ham, which can ruin your bayes filter.

Best regards,
Gerald

From rspamd at jubileegroup.co.uk  Wed Feb 28 19:16:31 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Wed, 28 Feb 2024 19:16:31 +0000 (GMT)
Subject: [Rspamd-Users] Multimap and syntax...
In-Reply-To: <5FDD5A1F-DF46-4E34-8761-7F10B0C96C5E@gcore.biz>
References: <03bdf4ff-f8c1-4706-9df1-862397d977d3@schani.com>
 <5FDD5A1F-DF46-4E34-8761-7F10B0C96C5E@gcore.biz>
Message-ID: <1e297b9-a34b-3864-9444-fe202b4a7ff7@jubileegroup.co.uk>

Hi there,

On Wed, 28 Feb 2024, Gerald Galster wrote:
> On Wed, 28 Feb 2024, christian via Users wrote:
> 
>> I use the filter email:domain:tld which according to Docs Somebody
>> "<user at foo.example.com> -> example.com " only returns the domain.
>> So I enter e.g. aok.de in my map. But what about: canford.co.uk? Is
>> co.uk then considered a TLD or as a domain and tld.

The documentation at

https://rspamd.com/doc/modules/multimap.html#from-rcpt-and-header-filters

is very confusing on these issues and your problem is understandable.

If you look at

https://rspamd.com/doc/modules/multimap.html#helo-hostname-filters

you'll see that the term 'tld' in the rspamd configuration is distinct
from the term 'top'.  To me this looks like an afterthought.  But even
if this were not so

> Rspamd includes the public suffix list (see https://publicsuffix.org/list/).
> https://github.com/rspamd/rspamd/blob/master/contrib/publicsuffix/effective_tld_names.dat

keeping lists like this current is an onerous task.  I wouldn't want
to (a) rely on that currency and (b) let my configuration be changed
without my approval when the list is updated - perhaps in ways which I
would not myself have chosen.

For example, looking at the .uk TLD, rspamd and Wikipedia disagree on
second level domains.  Compared with

https://en.wikipedia.org/wiki/.uk#Second-level_domains

the list at .../effective_tld_names.dat is missing at least these:

     .bl.uk ? used solely for the British Library
     .judiciary.uk ? judiciary of England and Wales
     .mod.uk ? armed forces and Ministry of Defence establishments and systems
     .nic.uk ? network use only (reserved exclusively for Nominet UK)
     .parliament.uk ? Parliament of the United Kingdom and the devolved national parliaments and assemblies
     .rct.uk ? used solely for the Royal Collection Trust
     .royal.uk ? used solely for the royal family website
     .ukaea.uk ? used solely for the United Kingdom Atomic Energy Authority

(The .sch.uk domain is debatable; it's given as "*.sch.uk" by Nominet/rspamd.)

My feeling is that in preference to "email:domain:tld" I might use
"email:domain" and decide for myself.  So I'd take on a maintenance
task, but at least I'd know who would be to blame when it all went
wrogn.

>> Should I do this with regex or not?
>
> With hyperscan enabled you can use lots of regexes without performance penalty.
> On the other hand you need to be familiar with regular expressions and be excact.
> Given the problems you currently have I don't recommend it because it's harder to debug.

Agreed.  I've used regexes almost daily for decades, and on occasion I
still find myself staring at one for hours before I finally figure out
what I've done wrong.  Sometimes you have to write code to debug them.
In addition it can take a bit of experience to avoid some pitfalls; if
you aren't careful you can easily craft a regex which will perform a
denial of service attack on your own system if somebody just sends a
big image file.

>> ... In my current white domain list I have around 2000
>> entries. Could it be that there are too many?
>
> Generally speaking, no.

Again I'd agree with Mr. Galster, rspamd can handle it, but I'd go
further and ask *why* do you have 2,000 entries?  For what I'm
thinking is a relatively new installation it seems to me like an awful
lot, and I wonder if that's a symptom of something.  Perhaps it's that
your spam rules are catching more than they really ought to?  If you
try to get around woolly spam rules which catch things that they
shouldn't catch by whitelisting everything then you're building
brokenness into your configuration.  Inevitably this becomes difficult
to cleanse other than by throwing away the baby with the bathwater.

> ... if you add e.g. adidas.com to your whitelist, any spammer that
> sends with @adidas.com is probably whitelisted due to score -20.

If you rely on the address in the 'From:' header, then unless you have
some other way of knowing that it's not forged you're more or less
obliged to check that it's vouched for by a DKIM signature.  This is
unlike the envelope 'from' address, which (apart, obviously, from all
the freemail domains) you can usually trust if SPF gives it the OK.
You'll find legitimate senders who can't get SPF right, but thesedays
their numbers are shrinking.

-- 

73,
Ged.

From list+rspamd at gcore.biz  Thu Feb 29 00:12:34 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Thu, 29 Feb 2024 01:12:34 +0100
Subject: [Rspamd-Users] Multimap and syntax...
In-Reply-To: <1e297b9-a34b-3864-9444-fe202b4a7ff7@jubileegroup.co.uk>
References: <03bdf4ff-f8c1-4706-9df1-862397d977d3@schani.com>
 <5FDD5A1F-DF46-4E34-8761-7F10B0C96C5E@gcore.biz>
 <1e297b9-a34b-3864-9444-fe202b4a7ff7@jubileegroup.co.uk>
Message-ID: <4D73B1C8-FBFF-4A92-B55B-31FEE9159BF3@gcore.biz>

>> Rspamd includes the public suffix list (see https://publicsuffix.org/list/).
>> https://github.com/rspamd/rspamd/blob/master/contrib/publicsuffix/effective_tld_names.dat
> 
> keeping lists like this current is an onerous task.  I wouldn't want
> to (a) rely on that currency and (b) let my configuration be changed
> without my approval when the list is updated - perhaps in ways which I
> would not myself have chosen.

The public suffix list is trustworthy and is used by major browsers
to restrict cookie setting or to decide if a user wants to open a
website URL or submit that text to a search engine.

https://publicsuffix.org/learn/

By using such a browser you implicitly rely on that list and accept changes.

As most top level domains are there to stay, the list is helpful even
if there are minor changes or additions. Consider for example the changes
for scw.cloud as a "top level domain": this is related to scaleway cloud
servers and from an rspamd perspective, scoring for scw.cloud as a second
level domain is probably sufficient (if not better then scoring for
individual third level customer*.scw.cloud domains). This is from a
technical / dns point of view where .cloud is the top level domain,
scw.cloud the second level domain and customer.scw.cloud the third level
domain.

In short, occasional modifications don't seem to change the bigger picture:
https://github.com/publicsuffix/list/commits/master/public_suffix_list.dat

On a browser level this is of course more important in order to avoid
cross origin attacks.


> For example, looking at the .uk TLD, rspamd and Wikipedia disagree on
> second level domains.  Compared with

This is probably because of different definitions, see
https://www.icann.org/en/system/files/files/octo-011-18may20-en.pdf

"The volunteers who run the PSL define a public suffix as: a domain under
which multiple parties that are unaffiliated with the operator of the domain
may register subdomains. ICANN?s Security and Stability Advisory Committee
(SSAC) defines a public suffix slightly differently: a domain that is
controlled by a public registry, under which subdomains can be registered
by third parties."

[...]

>> ... if you add e.g. adidas.com to your whitelist, any spammer that
>> sends with @adidas.com is probably whitelisted due to score -20.
> 
> If you rely on the address in the 'From:' header, then unless you have
> some other way of knowing that it's not forged you're more or less
> obliged to check that it's vouched for by a DKIM signature.  This is
> unlike the envelope 'from' address, which (apart, obviously, from all
> the freemail domains) you can usually trust if SPF gives it the OK.
> You'll find legitimate senders who can't get SPF right, but thesedays
> their numbers are shrinking.

I agree. Rspamd also helps with that:


    "WHITELIST_SPF_DKIM" = {
      valid_spf = true;
      valid_dkim = true;
      domains = [
        "https://maps.rspamd.com/rspamd/spf_dkim_whitelist.inc.zst",
        ...
      score = -3.0;	

spf_dkim_whitelist.inc contains:

# Domains with valid SPF and DKIM
[...]
airbnb.com
airtel.in
alibaba.com
aliexpress.com 2.0
alipay.com 2.0
allrecipes.com
amazon.ca
amazon.cn
amazon.co.jp
amazon.com
amazon.co.uk
amazon.de
...

Best regards,
Gerald


From rspamd at linuxmaker.com  Thu Feb 29 11:28:51 2024
From: rspamd at linuxmaker.com (Andreas)
Date: Thu, 29 Feb 2024 12:28:51 +0100
Subject: [Rspamd-Users] Control rspamd depending on subject content
In-Reply-To: <6012354.lOV4Wx5bFT@stuttgart>
References: <6012354.lOV4Wx5bFT@stuttgart>
Message-ID: <1889874.tdWV9SEqCh@stuttgart>

Am Samstag, 20. Januar 2024, 13:56:33 CET schrieb Gerald Galster:
> See https://rspamd.com/doc/modules/multimap.html
> 
> Example:
> 
> /etc/rspamd/local.d/multimap.conf
> 
> BLOCK_SUBJECT {
>         type = "header";
>         header = "Subject";
>         map = "https://or file:// [1]";
>         multi = true;
>         regexp = true;
>         prefilter = true;
>         action = "reject";
>         # message = "Spammy subject blocked";
>         # score = 1.0;
> }
> 
> --> action = "reject" will reject immediately, otherwise score will be added
> 
> 
> [1] Content of map file:
> # /regex/ SYMBOL:SCORE
> /Bitcoin-Effekt/ BLOCK_SUBJECT:4.5
> 
> 
> /etc/rspamd/local.d/metrics.conf
> 
> symbol "BLOCK_SUBJECT" {
>         weight = 1.0;
> }

I've now tried to implement it like this:
In /etc/rspamd/local.d/multimap.conf I have

BLOCK_SUBJECT {
         type = "header";
         header = "Subject";
         map = "file:///etc/rspamd/local.d/maps.d/banned_subjects.map";
         multi = true;
         regexp = true;
         prefilter = true;
         action = "reject";
}

inserted and created /etc/rspamd/local.d/maps.d/banned_subjects.map:

/*recipe-for-egg*/ BLOCK_SUBJECT:4.5

with the rights
-rw-r--r-- 1 _rspamd _rspamd 57 Feb 29 12:16 banned_subjects.map

additionally I have /etc/rspamd/local.d/metrics.conf

symbol "BLOCK_SUBJECT" {
         weight = 1.0;
}
created and then restarted the service rspamd.service.

However, emails with ?info prescription-free pharmacy? are still allowed 
through.

I would also be interested in the syntax
?map = ?file:///etc/rspamd/local.d/maps.d/banned_subjects.map?;?
must be or whether
?map = ?/etc/rspamd/local.d/maps.d/banned_subjects.map?;?
correct is?

Best regards

Andreas




From t.hendricks at interpool.de  Thu Feb 29 14:28:00 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Thu, 29 Feb 2024 15:28:00 +0100
Subject: [Rspamd-Users] Control rspamd depending on subject content
In-Reply-To: <1889874.tdWV9SEqCh@stuttgart>
References: <6012354.lOV4Wx5bFT@stuttgart> <1889874.tdWV9SEqCh@stuttgart>
Message-ID: <BF7D9DA5-AC6E-4683-97BF-19D6D1807D3D@interpool.de>

Hey Andreas,

all the multimaps I use are referenced like 

map = "${LOCAL_CONFDIR}/?
or
map = "${LOCAL_CONFDIR}/local.d/?

and work like charm.

Tino

> Am 29.02.2024 um 12:28 schrieb Andreas <rspamd at linuxmaker.com>:
> 
> Am Samstag, 20. Januar 2024, 13:56:33 CET schrieb Gerald Galster:
>> See https://rspamd.com/doc/modules/multimap.html
>> 
>> Example:
>> 
>> /etc/rspamd/local.d/multimap.conf
>> 
>> BLOCK_SUBJECT {
>>        type = "header";
>>        header = "Subject";
>>        map = "https://or file:// [1]";
>>        multi = true;
>>        regexp = true;
>>        prefilter = true;
>>        action = "reject";
>>        # message = "Spammy subject blocked";
>>        # score = 1.0;
>> }
>> 
>> --> action = "reject" will reject immediately, otherwise score will be added
>> 
>> 
>> [1] Content of map file:
>> # /regex/ SYMBOL:SCORE
>> /Bitcoin-Effekt/ BLOCK_SUBJECT:4.5
>> 
>> 
>> /etc/rspamd/local.d/metrics.conf
>> 
>> symbol "BLOCK_SUBJECT" {
>>        weight = 1.0;
>> }
> 
> I've now tried to implement it like this:
> In /etc/rspamd/local.d/multimap.conf I have
> 
> BLOCK_SUBJECT {
>         type = "header";
>         header = "Subject";
>         map = "file:///etc/rspamd/local.d/maps.d/banned_subjects.map";
>         multi = true;
>         regexp = true;
>         prefilter = true;
>         action = "reject";
> }
> 
> inserted and created /etc/rspamd/local.d/maps.d/banned_subjects.map:
> 
> /*recipe-for-egg*/ BLOCK_SUBJECT:4.5
> 
> with the rights
> -rw-r--r-- 1 _rspamd _rspamd 57 Feb 29 12:16 banned_subjects.map
> 
> additionally I have /etc/rspamd/local.d/metrics.conf
> 
> symbol "BLOCK_SUBJECT" {
>         weight = 1.0;
> }
> created and then restarted the service rspamd.service.
> 
> However, emails with ?info prescription-free pharmacy? are still allowed 
> through.
> 
> I would also be interested in the syntax
> ?map = ?file:///etc/rspamd/local.d/maps.d/banned_subjects.map?;?
> must be or whether
> ?map = ?/etc/rspamd/local.d/maps.d/banned_subjects.map?;?
> correct is?
> 
> Best regards
> 
> Andreas
> 
> 
> 
> -- 
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users


From rspamd at jubileegroup.co.uk  Thu Feb 29 17:19:22 2024
From: rspamd at jubileegroup.co.uk (G.W. Haywood)
Date: Thu, 29 Feb 2024 17:19:22 +0000 (GMT)
Subject: [Rspamd-Users] Control rspamd depending on subject content
In-Reply-To: <1889874.tdWV9SEqCh@stuttgart>
References: <6012354.lOV4Wx5bFT@stuttgart> <1889874.tdWV9SEqCh@stuttgart>
Message-ID: <11bb90f7-703e-6dde-cd9-befb9ff603@jubileegroup.co.uk>

Hi there,

On Thu, 29 Feb 2024, Andreas wrote:

> ... /etc/rspamd/local.d/maps.d/banned_subjects.map:
>
> /*recipe-for-egg*/ BLOCK_SUBJECT:4.5

You haven't used the 'i' modifier in the regexes which you've shown
to make them case insensitive.  That may be deliberate, but most of
the time I use it in, er, case the spammers use the 'shift' keys.

Be aware of the rules for constructing regexes.  They're a bit quirky.
Just as the character '*' is special in filename globs, it's special
in regexes too, but in a different way.  In a filename glob it more or
less means "anything".  In a regex, unless it is 'escaped', it means
"match if the character immediately preceding the asterisk is repeated
zero or more times".  I'm not sure your regex will do what you want it
to do.  Perhaps you mean something like

/.*recipe-for-egg.*/

There are plenty of regex tutorials on the Web, but when you search do
be aware that there are different types of regex:

https://en.wikipedia.org/wiki/Regular_expression#Syntax

They are mostly somewhat similar, with enough differences to make life
interesting if you use more than one kind of them.  The kind used by
rspamd is called "Perl Compatible Regular Expressions" (usually PCRE).
Perl's regexes are IMNSHO the best to use for more or less anything.
Avoid POSIX regexes if you can - I use them a lot and I wish I didn't
have to.

Even though rspamd uses PCRE, it has extended the syntax for its own
purposes.  The rspamd extensions let you specify exactly where to look
in the message for the match so it's much easier to avoid accidentally
matching something that you didn't mean to match - and it's also a lot
more efficient in terms of computing resources of course, you could be
searching just a single line instead of a huge image.  Look at

https://rspamd.com/doc/modules/regexp.html#regular-expressions

which shows you how you can identify with very good granularity the
part or parts of the message which you want to search.  For example
the 'Subject' header you could use

Subject=/egg/i{header}

looks *only* in the Subject header for the string ('egg' or 'Egg' or
'EGG' or 'eGG' or...).

Header field names are case insensitive according to the RFCs.

> However, emails with ?info prescription-free pharmacy? are still
> allowed through.

Maybe I'm missing something here but I think you need to show us more
of your config and generally give more details.  Which part of the
message are you searching for the text?  Is the search case sensitive?

This link might help:

https://jeffknerr.github.io/rspamd/regex/multimap/2021/03/02/rspamd-multimap-regex-examples.html

it was just a random result from my search using 'startpage.com' (the
Google front end which I prefer) for

"rspamd regex examples"

It looked like it made sense and the guy had made it work for him.  I
can't vouch for it, with more searching time I'm sure you could do a
lot better.

> I would also be interested in the syntax
> ?map = ?file:///etc/rspamd/local.d/maps.d/banned_subjects.map?;?
> must be or whether
> ?map = ?/etc/rspamd/local.d/maps.d/banned_subjects.map?;?
> correct is?

The use of both is shown at

https://rspamd.com/doc/modules/multimap.html#principles-of-work

Using a URI instead of using a file path changes the way in which the
content is accessed.  If there's no compelling reason to use a URI, I
would always use the simpler file path.

Have you used

rspamadm configtest

to check your configuration?

-- 

73,
Ged.

From rspamd at linuxmaker.com  Thu Feb 29 18:06:25 2024
From: rspamd at linuxmaker.com (Andreas)
Date: Thu, 29 Feb 2024 19:06:25 +0100
Subject: [Rspamd-Users] Control rspamd depending on subject content
In-Reply-To: <11bb90f7-703e-6dde-cd9-befb9ff603@jubileegroup.co.uk>
References: <6012354.lOV4Wx5bFT@stuttgart> <1889874.tdWV9SEqCh@stuttgart>
 <11bb90f7-703e-6dde-cd9-befb9ff603@jubileegroup.co.uk>
Message-ID: <4905962.31r3eYUQgx@stuttgart>

Am Donnerstag, 29. Februar 2024, 18:19:22 CET schrieb G.W. Haywood:
> Hi there,
> 
> On Thu, 29 Feb 2024, Andreas wrote:
> > ... /etc/rspamd/local.d/maps.d/banned_subjects.map:
> > 
> > /*recipe-for-egg*/ BLOCK_SUBJECT:4.5
> 
> You haven't used the 'i' modifier in the regexes which you've shown
> to make them case insensitive.  That may be deliberate, but most of
> the time I use it in, er, case the spammers use the 'shift' keys.
> 
> Be aware of the rules for constructing regexes.  They're a bit quirky.
> Just as the character '*' is special in filename globs, it's special
> in regexes too, but in a different way.  In a filename glob it more or
> less means "anything".  In a regex, unless it is 'escaped', it means
> "match if the character immediately preceding the asterisk is repeated
> zero or more times".  I'm not sure your regex will do what you want it
> to do.  Perhaps you mean something like
> 
> /.*recipe-for-egg.*/
> 
> There are plenty of regex tutorials on the Web, but when you search do
> be aware that there are different types of regex:
> 
> https://en.wikipedia.org/wiki/Regular_expression#Syntax
> 
> They are mostly somewhat similar, with enough differences to make life
> interesting if you use more than one kind of them.  The kind used by
> rspamd is called "Perl Compatible Regular Expressions" (usually PCRE).
> Perl's regexes are IMNSHO the best to use for more or less anything.
> Avoid POSIX regexes if you can - I use them a lot and I wish I didn't
> have to.
> 
> Even though rspamd uses PCRE, it has extended the syntax for its own
> purposes.  The rspamd extensions let you specify exactly where to look
> in the message for the match so it's much easier to avoid accidentally
> matching something that you didn't mean to match - and it's also a lot
> more efficient in terms of computing resources of course, you could be
> searching just a single line instead of a huge image.  Look at
> 
> https://rspamd.com/doc/modules/regexp.html#regular-expressions
> 
> which shows you how you can identify with very good granularity the
> part or parts of the message which you want to search.  For example
> the 'Subject' header you could use
> 
> Subject=/egg/i{header}
> 
> looks *only* in the Subject header for the string ('egg' or 'Egg' or
> 'EGG' or 'eGG' or...).
> 
> Header field names are case insensitive according to the RFCs.
> 
> > However, emails with ?info prescription-free pharmacy? are still
> > allowed through.
> 
> Maybe I'm missing something here but I think you need to show us more
> of your config and generally give more details.  Which part of the
> message are you searching for the text?  Is the search case sensitive?
> 
> This link might help:
> 
> https://jeffknerr.github.io/rspamd/regex/multimap/2021/03/02/rspamd-multimap
> -regex-examples.html
> 
> it was just a random result from my search using 'startpage.com' (the
> Google front end which I prefer) for
> 
> "rspamd regex examples"
> 
> It looked like it made sense and the guy had made it work for him.  I
> can't vouch for it, with more searching time I'm sure you could do a
> lot better.
> 
> > I would also be interested in the syntax
> > ?map = ?file:///etc/rspamd/local.d/maps.d/banned_subjects.map?;?
> > must be or whether
> > ?map = ?/etc/rspamd/local.d/maps.d/banned_subjects.map?;?
> > correct is?
> 
> The use of both is shown at
> 
> https://rspamd.com/doc/modules/multimap.html#principles-of-work
> 
> Using a URI instead of using a file path changes the way in which the
> content is accessed.  If there's no compelling reason to use a URI, I
> would always use the simpler file path.
> 
> Have you used
> 
> rspamadm configtest
> 
> to check your configuration?


Thanks to you,

that?s 
postfix/cleanup[2579711]: A196112007E: milter-reject: END-OF-MESSAGE from mail-
lj1-x236.google.com[2a00:1450:4864:20::236]: 5.7.1 Matched map: BLOCK_SUBJECT

what I want to see.
True, sometimes the devil is in the details. I overlooked that with the dots 
in the regexes.

The note with the variables for the path is also more my thing. In the docs 
and even in the config files
/* Example setup
sender_from_whitelist_user {
           type = "from";
           filter = "email:user";
           map = "file:///tmp/from.map";
           symbol = "SENDER_FROM_WHITELIST_USER";
           action = "accept"; # Prefilter mode
}*/
""file://...." is favored instead.
Anyway, thank you very much for your feedback.

Best regards

Andreas



From list+rspamd at gcore.biz  Thu Feb 29 19:27:41 2024
From: list+rspamd at gcore.biz (Gerald Galster)
Date: Thu, 29 Feb 2024 20:27:41 +0100
Subject: [Rspamd-Users] Control rspamd depending on subject content
In-Reply-To: <4905962.31r3eYUQgx@stuttgart>
References: <6012354.lOV4Wx5bFT@stuttgart> <1889874.tdWV9SEqCh@stuttgart>
 <11bb90f7-703e-6dde-cd9-befb9ff603@jubileegroup.co.uk>
 <4905962.31r3eYUQgx@stuttgart>
Message-ID: <1907D904-58C9-4FC9-B2AD-AA7E80959375@gcore.biz>

> that?s 
> postfix/cleanup[2579711]: A196112007E: milter-reject: END-OF-MESSAGE from mail-
> lj1-x236.google.com <http://lj1-x236.google.com/>[2a00:1450:4864:20::236]: 5.7.1 Matched map: BLOCK_SUBJECT
> 
> what I want to see.
> True, sometimes the devil is in the details. I overlooked that with the dots 
> in the regexes.

If you're unsure, test your regexes with perl (pcre = perl compatible regular expressions):

$ echo "info prescription-free pharmacy" | perl -ne '/pres.*?pharma/ ? print "MATCH\n" : print "NO_MATCH\n"'
MATCH

$ echo "info prescription-free pharmacy" | perl -ne '/pres.*?pharmaABCDEFG/ ? print "MATCH\n" : print "NO_MATCH\n"'
NO_MATCH

More about perl regular expressions:
https://perldoc.perl.org/perlre

Best regards,
Gerald

From t.hendricks at interpool.de  Thu Feb 29 20:11:47 2024
From: t.hendricks at interpool.de (Tino Hendricks)
Date: Thu, 29 Feb 2024 21:11:47 +0100
Subject: [Rspamd-Users] Control rspamd depending on subject content
In-Reply-To: <1907D904-58C9-4FC9-B2AD-AA7E80959375@gcore.biz>
References: <1907D904-58C9-4FC9-B2AD-AA7E80959375@gcore.biz>
Message-ID: <06B7A997-92B7-4A84-BBE4-22E2A61AF32B@interpool.de>

I fancy https://regexr.com/

T. 

> Am 29.02.2024 um 20:31 schrieb Gerald Galster <list+rspamd at gcore.biz>:
> 
> ?
>> 
>> that?s
>> postfix/cleanup[2579711]: A196112007E: milter-reject: END-OF-MESSAGE from mail-
>> lj1-x236.google.com <http://lj1-x236.google.com/>[2a00:1450:4864:20::236]: 5.7.1 Matched map: BLOCK_SUBJECT
>> 
>> what I want to see.
>> True, sometimes the devil is in the details. I overlooked that with the dots
>> in the regexes.
> 
> If you're unsure, test your regexes with perl (pcre = perl compatible regular expressions):
> 
> $ echo "info prescription-free pharmacy" | perl -ne '/pres.*?pharma/ ? print "MATCH\n" : print "NO_MATCH\n"'
> MATCH
> 
> $ echo "info prescription-free pharmacy" | perl -ne '/pres.*?pharmaABCDEFG/ ? print "MATCH\n" : print "NO_MATCH\n"'
> NO_MATCH
> 
> More about perl regular expressions:
> https://perldoc.perl.org/perlre
> 
> Best regards,
> Gerald
> --
> Users mailing list
> Users at lists.rspamd.com
> https://lists.rspamd.com/mailman/listinfo/users