From rspamd at linuxmaker.com Tue Apr 9 11:17:17 2024 From: rspamd at linuxmaker.com (Andreas) Date: Tue, 09 Apr 2024 13:17:17 +0200 Subject: [Rspamd-Users] RSpamd's autolearn function - Location in the file system Message-ID: <12415675.O9o76ZdvQC@stuttgart> Hello, Where are the values stored in RSpamd's autolearn function? I ask because I would like to make the learned patterns from an old server available on a new server. Thank you Andreas From list+rspamd at gcore.biz Wed Apr 10 14:28:05 2024 From: list+rspamd at gcore.biz (Gerald Galster) Date: Wed, 10 Apr 2024 16:28:05 +0200 Subject: [Rspamd-Users] RSpamd's autolearn function - Location in the file system In-Reply-To: <12415675.O9o76ZdvQC@stuttgart> References: <12415675.O9o76ZdvQC@stuttgart> Message-ID: <30D3B19A-3FA4-4DB7-9E43-37CC3A0FE2D6@gcore.biz> > Where are the values stored in RSpamd's autolearn function? I ask because I Usually redis (backend). > would like to make the learned patterns from an old server available on a new > server. https://rspamd.com/doc/modules/bayes_expiry.html#limiting-memory-usage-to-a-fixed-amount Do you have a separate redis instance for bayes? Then you can copy the .rdb file to the new server. Redis dumps are often stored in /var/lib/redis/.../dump.rdb, depending on your distribution. Best regards, Gerald From rspamd at linuxmaker.com Wed Apr 10 14:57:21 2024 From: rspamd at linuxmaker.com (Andreas) Date: Wed, 10 Apr 2024 16:57:21 +0200 Subject: [Rspamd-Users] RSpamd's autolearn function - Location in the file system In-Reply-To: <30D3B19A-3FA4-4DB7-9E43-37CC3A0FE2D6@gcore.biz> References: <12415675.O9o76ZdvQC@stuttgart> <30D3B19A-3FA4-4DB7-9E43-37CC3A0FE2D6@gcore.biz> Message-ID: <12433489.O9o76ZdvQC@stuttgart> Am Mittwoch, 10. April 2024, 16:28:05 CEST schrieb Gerald Galster: > > Where are the values stored in RSpamd's autolearn function? I ask because > > I > > Usually redis (backend). > > > would like to make the learned patterns from an old server available on a > > new server. > > https://rspamd.com/doc/modules/bayes_expiry.html#limiting-memory-usage-to-a-> fixed-amount > > Do you have a separate redis instance for bayes? Then you can copy the .rdb > file to the new server. > > Redis dumps are often stored in /var/lib/redis/.../dump.rdb, depending on > your distribution. > > Best regards, > Gerald Thank you Gerald, your information is very helpful. Best regards Andreas From lists at at.encryp.ch Wed Apr 10 20:39:54 2024 From: lists at at.encryp.ch (Serhii) Date: Wed, 10 Apr 2024 20:39:54 +0000 Subject: [Rspamd-Users] Return-Path validation Message-ID: <7370616d-998c044c-25ab-4508-867a-4529e10ff7d3-at.encryp.ch-74726170@at.encryp.ch> Hello, Currently my setup involves Postfix and rspamd. The milter interface is used. It seems that doing sender address validation is extremely useful to discard spam. However, the issue that currently MAIL FROM reachability is checked by postfix before rspamd decides if SPF is allowed. My question is it possible to use Lua to acomplish Return-Path validation for scoring purpose, but only if SPF is not fail/softfail? Thanks. -- Send unsolicited bulk mail to carle34 at at.encryp.ch From rspamd at jubileegroup.co.uk Wed Apr 10 21:44:33 2024 From: rspamd at jubileegroup.co.uk (G.W. Haywood) Date: Wed, 10 Apr 2024 22:44:33 +0100 (BST) Subject: [Rspamd-Users] Return-Path validation In-Reply-To: <7370616d-998c044c-25ab-4508-867a-4529e10ff7d3-at.encryp.ch-74726170@at.encryp.ch> References: <7370616d-998c044c-25ab-4508-867a-4529e10ff7d3-at.encryp.ch-74726170@at.encryp.ch> Message-ID: <149743-4068-4bb0-c065-0455a8d446@jubileegroup.co.uk> Hi there, On Wed, 10 Apr 2024, Serhii via Users wrote: > ... It seems that doing sender address validation is extremely useful ... Assuming that you mean callback verification: https://en.wikipedia.org/wiki/Callback_verification then the usefulness of the technique is limited, and I believe that the majority considers it to be abusive. I certainly do. -- 73, Ged. From lists at at.encryp.ch Thu Apr 11 05:18:00 2024 From: lists at at.encryp.ch (Serhii) Date: Thu, 11 Apr 2024 05:18:00 +0000 (UTC) Subject: [Rspamd-Users] Return-Path validation In-Reply-To: <149743-4068-4bb0-c065-0455a8d446@jubileegroup.co.uk> References: <7370616d-998c044c-25ab-4508-867a-4529e10ff7d3-at.encryp.ch-74726170@at.encryp.ch> <149743-4068-4bb0-c065-0455a8d446@jubileegroup.co.uk> Message-ID: <7370616d-c33eacef-54cb-4527-b95c-846bea157b9c-at.encryp.ch-74726170@at.encryp.ch> Yes, I meant exactly callback verification. Have used this technique for a while already, surprised it is considered abusive. Thanks for sharing this information, I will reconsider using it. 2024-04-10T22:24:04Z G.W. Haywood : > Assuming that you mean callback verification: > > https://en.wikipedia.org/wiki/Callback_verification > > then the usefulness of the technique is limited, and I believe that > the majority considers it to be abusive.? I certainly do. -- Send unsolicited bulk email to carle34 at at.encryp.ch From lists at at.encryp.ch Thu Apr 11 11:23:36 2024 From: lists at at.encryp.ch (Serhii) Date: Thu, 11 Apr 2024 11:23:36 +0000 Subject: [Rspamd-Users] Integrate Rspamd with submit.spamhaus.org Message-ID: <7370616d-b7f40bf0-2d77-4231-a19f-631da0f904a8-at.encryp.ch-74726170@at.encryp.ch> Hello, Has anyone already implemented integration for submit.spamhaus.org[1] with Rspamd? I have a bunch of personal spam traps and would like to make them more useful. Thanks! [1]: -- Send unsolicited bulk mail to carle34 at at.encryp.ch From trashcan at ellael.org Thu Apr 11 12:04:08 2024 From: trashcan at ellael.org (Michael Grimm) Date: Thu, 11 Apr 2024 14:04:08 +0200 Subject: [Rspamd-Users] syslog and missing log messages Message-ID: Hi, this is rspamd 3.8.4 and postfix 3.9.0 (milter) running in a jail with FreeBSD 14-STABLE as host. My /etc/syslog.conf is configured to send all syslog messages to the host's syslog, *and* to a logfile in addition: mail.* /var/log/maillog *.* @ My local.d/loggin.inc is as follows: type = "syslog"; facility = "mail"; level = "info"; # log all non-debug messages I do have difficulties to understand, why some the rspamd messages aren't forwarded to the host's syslog but are reported to /var/log/maillog. Example /var/log/maillog in the jail: Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; finalize_item: slow rule: DKIM_CHECK(189): 803.19 ms; enable slow timer delay Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; rspamd_stat_classifiers_process: skip statistics as SPAM class is missing Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; rspamd_task_write_log: id: , qid: <4VFd8q0HCjzxrc>, ip: ... Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; rspamd_protocol_http_reply: regexp statistics: 225 pcre regexps scanned, 1 regexps matched, 176 regexps total, 11 regexps cached, 46.92KiB scanned using pcre, 46.92KiB scanned total Apr 11 13:42:20 mail rspamd[72887]: ; proxy; proxy_milter_finish_handler: finished milter connection Apr 11 13:43:01 mail rspamd[72888]: ; map; http_map_finish: data is not modified for server maps.rspamd.com, next check at Thu, 11 Apr 2024 15:43:01 GMT (http cache based: Thu, 11 Apr 2024 15:43:01 GMT) Corresponding syslog at the host: Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; finalize_item: slow rule: DKIM_CHECK(189): 803.19 ms; enable slow timer delay Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; rspamd_stat_classifiers_process: skip statistics as SPAM class is missing ? rspamd_task_write_log ? rspamd_protocol_http_reply Apr 11 13:42:20 mail rspamd[72887]: ; proxy; proxy_milter_finish_handler: finished milter connection Apr 11 13:43:01 mail rspamd[72888]: ; map; http_map_finish: data is not modified for server maps.rspamd.com, next check at Thu, 11 Apr 2024 15:43:01 GMT (http cache based: Thu, 11 Apr 2024 15:43:01 GMT) FYI: reverting /etc/syslog.conf entries and omitting 'mail.* /var/log/maillog' doesn't help. Any hints for understanding and thus solving this issue is highly appreciated. Thanks in advance and regards, Michael From rspamd at jubileegroup.co.uk Thu Apr 11 13:31:11 2024 From: rspamd at jubileegroup.co.uk (G.W. Haywood) Date: Thu, 11 Apr 2024 14:31:11 +0100 (BST) Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: References: Message-ID: <7f462245-8946-75c-a6e9-c89e4ca0399@jubileegroup.co.uk> Hi there, On Thu, 11 Apr 2024, Michael Grimm via Users wrote: > ... I do have difficulties to understand, why some the rspamd > messages aren't forwarded to the host's syslog but are reported to > /var/log/maillog. ... Sending syslog messages over the network means you will be using UDP. If you use UDP you cannot guarantee that syslog will log everything it is asked to log. That's because UDP connections are not "reliable" in the sense that TCP connections are described as "reliable". A message sent by UDP might arrive at the destination, or it might not (for many possible reasons), and the sender has no way of knowing what happened. That being said, if the hardware resources are not over-committed then most of the time you should get away with it. Perhaps there are some performance issues in your system? The usual way to be sure that messages are logged is to write them to files, but obviously in a jail you're limited to accessible files. -- 73, Ged. From list+rspamd at gcore.biz Thu Apr 11 14:39:16 2024 From: list+rspamd at gcore.biz (Gerald Galster) Date: Thu, 11 Apr 2024 16:39:16 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: References: Message-ID: <9858DC9F-DBBD-4128-A33A-DC7D3C895227@gcore.biz> > this is rspamd 3.8.4 and postfix 3.9.0 (milter) running in a jail with FreeBSD 14-STABLE as host. > > My /etc/syslog.conf is configured to send all syslog messages to the host's syslog, *and* to a logfile in addition: > [...] > I do have difficulties to understand, why some the rspamd messages aren't forwarded to the host's syslog but are reported to /var/log/maillog. Just another guess ... Depending on your mailvolume rspamd can produce a lot of logs. Some syslog implementations have rate limits and drop messages. Best regards, Gerald From list+rspamd at gcore.biz Thu Apr 11 14:44:25 2024 From: list+rspamd at gcore.biz (Gerald Galster) Date: Thu, 11 Apr 2024 16:44:25 +0200 Subject: [Rspamd-Users] Return-Path validation In-Reply-To: <7370616d-c33eacef-54cb-4527-b95c-846bea157b9c-at.encryp.ch-74726170@at.encryp.ch> References: <7370616d-998c044c-25ab-4508-867a-4529e10ff7d3-at.encryp.ch-74726170@at.encryp.ch> <149743-4068-4bb0-c065-0455a8d446@jubileegroup.co.uk> <7370616d-c33eacef-54cb-4527-b95c-846bea157b9c-at.encryp.ch-74726170@at.encryp.ch> Message-ID: > Yes, I meant exactly callback verification. > > Have used this technique for a while already, surprised it is considered abusive. > > Thanks for sharing this information, I will reconsider using it. As you are using postfix: https://www.postfix.org/ADDRESS_VERIFICATION_README.html "Some sites may denylist you when you are probing them too often (a probe is an SMTP session that does not deliver mail), or when you are probing them too often for a non-existent address. This is one reason why you should use sender address verification sparingly, if at all, when your site receives lots of email." Best regards, Gerald From trashcan at ellael.org Thu Apr 11 15:14:11 2024 From: trashcan at ellael.org (Michael Grimm) Date: Thu, 11 Apr 2024 17:14:11 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: <7f462245-8946-75c-a6e9-c89e4ca0399@jubileegroup.co.uk> References: <7f462245-8946-75c-a6e9-c89e4ca0399@jubileegroup.co.uk> Message-ID: <19728A1C-BE08-436C-A996-71292B4E9924@ellael.org> G.W. Haywood wrote: > Sending syslog messages over the network means you will be using UDP. > > If you use UDP you cannot guarantee that syslog will log everything it > is asked to log. That's because UDP connections are not "reliable" in > the sense that TCP connections are described as "reliable". A message > sent by UDP might arrive at the destination, or it might not (for many > possible reasons), and the sender has no way of knowing what happened. > That being said, if the hardware resources are not over-committed then > most of the time you should get away with it. Perhaps there are some > performance issues in your system? No, my system is bored to death ;-) Small server for a handful users with moderate usage. Nothing fancy. But your answer let me think about UDP limitations. Thus, I did check if the sizes of all missing messages are exceeding RFC 3164 length of 1028 bytes, but no, they don't. Next, I did increase verbosity of syslogd (-v -v) and did test syslog forwarding with logger: logger -p mail.info < missing_line That worked, and now the receiving syslogd at the host reports both facility *and* level: Apr 11 17:07:11 mail rspamd[3653]: ; proxy; rspamd_task_write_log: id: , qid: <4VFjjC4KXQzkp1>, ip: ? Ok, my missing lines are of level 'debug'. Thus I increased the level in logging.inc to "debug", but to now avail. Now, the logfile in the jail collects a lot of messages of level "debug", but none of these messages are sent to the host. It looks to me as if rspamd limits syslog messages to levels below "debug". > The usual way to be sure that messages are logged is to write them to > files, but obviously in a jail you're limited to accessible files. Yeah, and sockets. Thanks and regards, Michael From trashcan at ellael.org Thu Apr 11 15:38:58 2024 From: trashcan at ellael.org (Michael Grimm) Date: Thu, 11 Apr 2024 17:38:58 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: <9858DC9F-DBBD-4128-A33A-DC7D3C895227@gcore.biz> References: <9858DC9F-DBBD-4128-A33A-DC7D3C895227@gcore.biz> Message-ID: <42B4C837-05CC-410F-8466-15DB6B1C484E@ellael.org> Gerald Galster wrote: >> this is rspamd 3.8.4 and postfix 3.9.0 (milter) running in a jail with FreeBSD 14-STABLE as host. >> >> My /etc/syslog.conf is configured to send all syslog messages to the host's syslog, *and* to a logfile in addition: >> [...] >> I do have difficulties to understand, why some the rspamd messages aren't forwarded to the host's syslog but are reported to /var/log/maillog. > > Just another guess ... > > Depending on your mailvolume rspamd can produce a lot of logs. > Some syslog implementations have rate limits and drop messages. Both servers of mine are used by a handful users, and CPUs are bored all time ;-) Thanks and regards, Michael From lists at at.encryp.ch Thu Apr 11 16:06:56 2024 From: lists at at.encryp.ch (Serhii) Date: Thu, 11 Apr 2024 16:06:56 +0000 (UTC) Subject: [Rspamd-Users] Return-Path validation In-Reply-To: References: <7370616d-998c044c-25ab-4508-867a-4529e10ff7d3-at.encryp.ch-74726170@at.encryp.ch> <149743-4068-4bb0-c065-0455a8d446@jubileegroup.co.uk> <7370616d-c33eacef-54cb-4527-b95c-846bea157b9c-at.encryp.ch-74726170@at.encryp.ch> Message-ID: <7370616d-0d96d8b9-8505-4955-83a0-cf048680f063-at.encryp.ch-74726170@at.encryp.ch> The thing is I've encountered numerous sender address verification probes from other systems against me, including services dedicated to that purpose, and thought it is not a controversial activity. Anyway, thanks for heads up. -- Send unsolicited bulk email to carle34 at at.encryp.ch From list+rspamd at gcore.biz Thu Apr 11 17:58:02 2024 From: list+rspamd at gcore.biz (Gerald Galster) Date: Thu, 11 Apr 2024 19:58:02 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: References: Message-ID: <86065622-D903-43C5-872B-5867BD9894DE@gcore.biz> > My local.d/loggin.inc is as follows: > > type = "syslog"; > facility = "mail"; > level = "info"; # log all non-debug messages I have the same config (but running linux and systemd-journald) and I get SYSLOG_FACILITY=2 (mail system) PRIORITY=7 (debug) MESSAGE=; proxy; rspamd_task_write_log: id: .... So it basically works. Does your syslog ignore debug for mail? What happens if you change the facility to something else that might not have any defaults associated with it, like # local.d/logging.inc facility = "local5"; ... Best regards, Gerald From rspamd at vlh.dk Thu Apr 11 18:57:39 2024 From: rspamd at vlh.dk (rspamd at vlh.dk) Date: Thu, 11 Apr 2024 20:57:39 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: References: Message-ID: <000101da8c42$20802490$61806db0$@vlh.dk> Hi, The logs on the ssyslog end seems to have some data in it that's not in the jail - specifically this part: ? rspamd_task_write_log ? rspamd_protocol_http_reply Could transmission from jail to syslog host have gone wrong? -Kim > -----Original Message----- > From: Users On Behalf Of Michael Grimm > via Users > Sent: 11. april 2024 14:04 > To: users at lists.rspamd.com > Cc: Michael Grimm > Subject: [Rspamd-Users] syslog and missing log messages > > Hi, > > this is rspamd 3.8.4 and postfix 3.9.0 (milter) running in a jail with FreeBSD > 14-STABLE as host. > > My /etc/syslog.conf is configured to send all syslog messages to the host's > syslog, *and* to a logfile in addition: > > mail.* /var/log/maillog > *.* @ > > My local.d/loggin.inc is as follows: > > type = "syslog"; > facility = "mail"; > level = "info"; # log all non-debug messages > > I do have difficulties to understand, why some the rspamd messages aren't > forwarded to the host's syslog but are reported to /var/log/maillog. > > Example /var/log/maillog in the jail: > > Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; finalize_item: slow > rule: DKIM_CHECK(189): 803.19 ms; enable slow timer delay Apr 11 13:42:20 > mail rspamd[72887]: <358183>; proxy; rspamd_stat_classifiers_process: skip > statistics as SPAM class is missing Apr 11 13:42:20 mail rspamd[72887]: > <358183>; proxy; rspamd_task_write_log: id: > , qid: > <4VFd8q0HCjzxrc>, ip: ... > Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; > rspamd_protocol_http_reply: regexp statistics: 225 pcre regexps scanned, 1 > regexps matched, 176 regexps total, 11 regexps cached, 46.92KiB scanned > using pcre, 46.92KiB scanned total Apr 11 13:42:20 mail rspamd[72887]: > ; proxy; proxy_milter_finish_handler: finished milter connection Apr > 11 13:43:01 mail rspamd[72888]: ; map; http_map_finish: data is > not modified for server maps.rspamd.com, next check at Thu, 11 Apr 2024 > 15:43:01 GMT (http cache based: Thu, 11 Apr 2024 15:43:01 GMT) > > Corresponding syslog at the host: > > Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; > finalize_item: slow rule: DKIM_CHECK(189): 803.19 ms; enable slow timer > delay Apr 11 13:42:20 mail rspamd[72887]: <358183>; proxy; > rspamd_stat_classifiers_process: skip statistics as SPAM class is missing ? > rspamd_task_write_log ? rspamd_protocol_http_reply Apr 11 13:42:20 > mail rspamd[72887]: ; proxy; > proxy_milter_finish_handler: finished milter connection Apr 11 13:43:01 > mail rspamd[72888]: ; map; http_map_finish: data is not > modified for server maps.rspamd.com, next check at Thu, 11 Apr 2024 > 15:43:01 GMT (http cache based: Thu, 11 Apr 2024 15:43:01 GMT) > > > FYI: reverting /etc/syslog.conf entries and omitting 'mail.* /var/log/maillog' > doesn't help. > > > Any hints for understanding and thus solving this issue is highly appreciated. > > Thanks in advance and regards, > Michael > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users From trashcan at ellael.org Thu Apr 11 19:09:26 2024 From: trashcan at ellael.org (Michael Grimm) Date: Thu, 11 Apr 2024 21:09:26 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: <86065622-D903-43C5-872B-5867BD9894DE@gcore.biz> References: <86065622-D903-43C5-872B-5867BD9894DE@gcore.biz> Message-ID: Gerald Galster wrote: >> My local.d/loggin.inc is as follows: >> >> type = "syslog"; >> facility = "mail"; >> level = "info"; # log all non-debug messages > > I have the same config (but running linux and systemd-journald) and I get > > SYSLOG_FACILITY=2 (mail system) > PRIORITY=7 (debug) > MESSAGE=; proxy; rspamd_task_write_log: id: .... > > So it basically works. Good to know. > Does your syslog ignore debug for mail? Bingo! Many, many kudos! I had had in my over 20+ years grown /etc/syslog.conf: *.info;console.notice;authpriv.none;kern.debug /var/log/messages mail.info /var/log/maillog So, neither ... > What happens if you change the facility to something else that > might not have any defaults associated with it, like > > # local.d/logging.inc > facility = "local5"; ? local5.debug nor mail.debug ended in either /var/log/messages or /var/log/maillog :-( Thus, it is time for a Fr?hjahrsputz [1] in my /etc/syslog But there is one more puzzle remaining, namely why my local.d/loggin.inc ? type = "syslog"; facility = "mail"; level = "info"; # log all non-debug messages ? will send mail.*debug* messages to my host's syslog in the first place? See: Apr 11 20:58:50 mail rspamd[16488]: <9f588c>; proxy; rspamd_task_write_log: id: , qid: , ip: ... Thank you very much for your eye-opener, very much appreciated. Regards, Michael [1] I tend to translate that to "spring-clean" ;-) From list+rspamd at gcore.biz Fri Apr 12 01:23:06 2024 From: list+rspamd at gcore.biz (Gerald Galster) Date: Fri, 12 Apr 2024 03:23:06 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: References: <86065622-D903-43C5-872B-5867BD9894DE@gcore.biz> Message-ID: > But there is one more puzzle remaining, namely why my local.d/loggin.inc ? > > type = "syslog"; > facility = "mail"; > level = "info"; # log all non-debug messages > > ? will send mail.*debug* messages to my host's syslog in the first place? local.d/logging.inc level "info" is not necessarily syslog level info. src/libserver/logger.h: #define msg_notice(...) rspamd_default_log_function(G_LOG_LEVEL_MESSAGE, #define msg_info(...) rspamd_default_log_function(G_LOG_LEVEL_INFO rspamd_task_write_log calls msg_notice_task and probably uses G_LOG_LEVEL_MESSAGE. https://github.com/rspamd/rspamd/blob/master/src/libserver/logger/logger_syslog.c#L58-L82 Log levels G_LOG_LEVEL_DEBUG, _INFO, _WARNING, _ERR are mapped at line 66+. G_LOG_LEVEL_MESSAGE is not part of levels_match[] (line 69+), so it should default to syslog_level = LOG_DEBUG in line 82, which would explain why you see mail.*debug* ... or my quick code grep is bogus and it works totally different :-) Best regards, Gerald From trashcan at ellael.org Fri Apr 12 10:19:19 2024 From: trashcan at ellael.org (Michael Grimm) Date: Fri, 12 Apr 2024 12:19:19 +0200 Subject: [Rspamd-Users] syslog and missing log messages In-Reply-To: References: <86065622-D903-43C5-872B-5867BD9894DE@gcore.biz> Message-ID: <3B9AB47D-A15D-4FB5-BFA9-4BE696014195@ellael.org> Gerald Galster wrote: >> But there is one more puzzle remaining, namely why my local.d/loggin.inc ? >> >> type = "syslog"; >> facility = "mail"; >> level = "info"; # log all non-debug messages >> >> ? will send mail.*debug* messages to my host's syslog in the first place? > > local.d/logging.inc level "info" is not necessarily syslog level info. > > src/libserver/logger.h: > #define msg_notice(...) rspamd_default_log_function(G_LOG_LEVEL_MESSAGE, > #define msg_info(...) rspamd_default_log_function(G_LOG_LEVEL_INFO > > rspamd_task_write_log calls msg_notice_task and probably uses G_LOG_LEVEL_MESSAGE. > > > https://github.com/rspamd/rspamd/blob/master/src/libserver/logger/logger_syslog.c#L58-L82 > > Log levels G_LOG_LEVEL_DEBUG, _INFO, _WARNING, _ERR are mapped at line 66+. > G_LOG_LEVEL_MESSAGE is not part of levels_match[] (line 69+), so it should default > to syslog_level = LOG_DEBUG in line 82, which would explain why you see mail.*debug* > ... or my quick code grep is bogus and it works totally different :-) I do follow your analysis, and do have the feeling, that this isn't a feature ;-) And, I do not understand that LOG_DEBUG is part of levels_match[] although syslog_level is set to LOG_DEBUG as default, before looking for a different syslog level in levels_match[]? It seems to me that G_LOG_LEVEL_MESSAGE should replace G_LOG_LEVEL_DEBUG in levels_match[]. Most probably G_LOG_LEVEL_MESSAGE should match LOG_INFO, because I cannot find a single LOG_NOTICE or LOG_MESSAGE reference in the source code. Regards, Michael From riccardo.alfieri at spamteq.com Mon Apr 15 08:48:58 2024 From: riccardo.alfieri at spamteq.com (Riccardo Alfieri) Date: Mon, 15 Apr 2024 08:48:58 +0000 Subject: [Rspamd-Users] Enhancements to Spamhaus plugin - Hash Blocklist update Message-ID: Hello Rspamd users, We?re pleased to share a new version of Spamhaus? plugin is available to the Rspamd community. This incorporates the latest enhancement to one of Spamhaus? Content Blocklists - the Hash Blocklist (HBL). This version also resolves a memory issue observed in an earlier February release. *Enhancements to the HBL* The Hash Blocklist has gained a new subset ? URLs. This subset will include any URLs observed to be malicious or suspicious by Spamhaus. This is in addition to the existing subsets: compromised email addresses, cryptowallets, and malware files. All the relevant technical documentation is available at: https://docs.spamhaus.com/datasets/docs/source/10-data-type-documentation/datasets/030-datasets.html#hbl *How to access the updates* To?ensure you?are using the latest version of the Rspamd plugin, including the enhanced HBL, please update by following the installation instructions at the URL below: https://github.com/spamhaus/rspamd-dqs *A final ?thank you? to SURBL* For those looking to combine data sources, thank you to SURBL for supporting Spamhaus to create this enhancement as a compatible data solution. Kind regards, -- Best regards, Riccardo Alfieri Spamhaus Technology https://www.spamhaus.com/ From rspamd at linuxmaker.com Mon Apr 15 12:24:59 2024 From: rspamd at linuxmaker.com (Andreas) Date: Mon, 15 Apr 2024 14:24:59 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd Message-ID: <6041070.lOV4Wx5bFT@stuttgart> Hello, I have a question about blocking emails with file attachments. I saw that there is mime_types.conf in /etc/rspamd/local.d which already contains file extensions like exe, jar. I would like to have Rspamd block emails that contain the file extensions doc, docx, xls, xlsx in the attachment. Whether this makes sense should not be a criterion here. We don't use Microsoft here and therefore don't need any external MS files, especially since this content could also be sent as a PDF. So how can I specifically block emails containing Word and Excel files? I had in the mime_types.conf ppt = 3; doc = 3; xls = 3; pptx = 3; docx = 3; xlsx = 3; added, but that doesn't work. Emails with such file attachments still go through unhindered. Best regards Andreas From florian at effenberger.org Mon Apr 15 12:45:02 2024 From: florian at effenberger.org (Florian Effenberger) Date: Mon, 15 Apr 2024 14:45:02 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: <6041070.lOV4Wx5bFT@stuttgart> References: <6041070.lOV4Wx5bFT@stuttgart> Message-ID: <75a7f40b-302e-4aa9-9ffd-787b206c8549@effenberger.org> Hi, Andreas wrote on 15.04.24 at 14:24: > I have a question about blocking emails with file attachments. I saw that there > is mime_types.conf in /etc/rspamd/local.d which already contains file > extensions like exe, jar. I think it should work with the following in local.d/multimap.conf: FILENAME_BLACKLISTED { type = "filename"; filter = "extension"; map = "$CONFDIR/local.d/filename.map"; action = "reject"; message = "A restricted file type was found"; skip_archives = true; } filename.map itself only contains the suffix of the files to be blocked, e.g. vbs dll exe com Florian From rspamd at jubileegroup.co.uk Mon Apr 15 13:38:22 2024 From: rspamd at jubileegroup.co.uk (G.W. Haywood) Date: Mon, 15 Apr 2024 14:38:22 +0100 (BST) Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: <6041070.lOV4Wx5bFT@stuttgart> References: <6041070.lOV4Wx5bFT@stuttgart> Message-ID: <82c6ecd-e986-4c2a-db7a-33c98bd64497@jubileegroup.co.uk> Hi there, On Mon, 15 Apr 2024, Andreas wrote: > ... > I had in the mime_types.conf > ppt = 3; > ... > docx = 3; > xlsx = 3; > added, but that doesn't work. Did you restart rspamd after making your changes? -- 73, Ged. From rspamd at linuxmaker.com Mon Apr 15 13:46:56 2024 From: rspamd at linuxmaker.com (Andreas) Date: Mon, 15 Apr 2024 15:46:56 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: <75a7f40b-302e-4aa9-9ffd-787b206c8549@effenberger.org> References: <6041070.lOV4Wx5bFT@stuttgart> <75a7f40b-302e-4aa9-9ffd-787b206c8549@effenberger.org> Message-ID: <1889291.tdWV9SEqCh@stuttgart> Am Montag, 15. April 2024, 14:45:02 CEST schrieb Florian Effenberger: > I think it should work with the following in local.d/multimap.conf: > > FILENAME_BLACKLISTED { > type = "filename"; > filter = "extension"; > map = "$CONFDIR/local.d/filename.map"; > action = "reject"; > message = "A restricted file type was found"; > skip_archives = true; > } > > filename.map itself only contains the suffix of the files to be blocked, > e.g. > > vbs > dll > exe > com > > Florian Fine, it worked. What is mime_types.conf used for then? Greetings and thanks Andreas From rspamd at linuxmaker.com Mon Apr 15 14:30:58 2024 From: rspamd at linuxmaker.com (Andreas) Date: Mon, 15 Apr 2024 16:30:58 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: <82c6ecd-e986-4c2a-db7a-33c98bd64497@jubileegroup.co.uk> References: <6041070.lOV4Wx5bFT@stuttgart> <82c6ecd-e986-4c2a-db7a-33c98bd64497@jubileegroup.co.uk> Message-ID: <2937265.e9J7NaK4W3@stuttgart> Am Montag, 15. April 2024, 15:38:22 CEST schrieb G.W. Haywood: > Did you restart rspamd after making your changes? Always, because it is standard at all Linux and Unix OS for reloading or restarting the preferred service. From florian at effenberger.org Mon Apr 15 15:02:36 2024 From: florian at effenberger.org (Florian Effenberger) Date: Mon, 15 Apr 2024 17:02:36 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: <1889291.tdWV9SEqCh@stuttgart> References: <6041070.lOV4Wx5bFT@stuttgart> <75a7f40b-302e-4aa9-9ffd-787b206c8549@effenberger.org> <1889291.tdWV9SEqCh@stuttgart> Message-ID: Hello, Andreas wrote on 15.04.24 at 15:46: > Fine, > it worked. What is mime_types.conf used for then? I don't know - I only know the variant I sent to you, never tried the mime_types.conf approach. Florian From a.wass at glas-gasperlmair.at Tue Apr 16 05:52:51 2024 From: a.wass at glas-gasperlmair.at (Andreas Wass - Glas Gasperlmair) Date: Tue, 16 Apr 2024 07:52:51 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: References: <6041070.lOV4Wx5bFT@stuttgart> <75a7f40b-302e-4aa9-9ffd-787b206c8549@effenberger.org> <1889291.tdWV9SEqCh@stuttgart> Message-ID: <43c03fa0-1070-460a-9cfa-fe3f003c600e@glas-gasperlmair.at> i use it like this: vi /etc/rspamd/local.d/mime_types.conf # Extensions that are treated as 'bad' # Number is score multiply factor bad_extensions = { ? accdb = 1, ? accdr = 1, ? ace = 1000, ? ade = 1, ? adp = 1, ? ani = 1, ? app = 1, ? arj = 1, ? asd = 1, ? asf = 1, ? asx = 1, ? b64 = 1, ? bas = 1, ? bat = 1000, tag your bad extensions with a high multipicator (1000) next: systemctl reload rspamd best regards, andi Am 15.04.2024 um 17:02 schrieb Florian Effenberger: > Hello, > > Andreas wrote on 15.04.24 at 15:46: >> Fine, >> it worked. What is mime_types.conf used for then? > > I don't know - I only know the variant I sent to you, never tried the > mime_types.conf approach. > > Florian From rspamd at linuxmaker.com Tue Apr 16 06:58:00 2024 From: rspamd at linuxmaker.com (Andreas) Date: Tue, 16 Apr 2024 08:58:00 +0200 Subject: [Rspamd-Users] Block emails with .doc, .xls attachments with Rspamd In-Reply-To: <43c03fa0-1070-460a-9cfa-fe3f003c600e@glas-gasperlmair.at> References: <6041070.lOV4Wx5bFT@stuttgart> <43c03fa0-1070-460a-9cfa-fe3f003c600e@glas-gasperlmair.at> Message-ID: <1887886.tdWV9SEqCh@stuttgart> Am Dienstag, 16. April 2024, 07:52:51 CEST schrieb Andreas Wass - Glas Gasperlmair: > i use it like this: > > vi /etc/rspamd/local.d/mime_types.conf > > # Extensions that are treated as 'bad' > # Number is score multiply factor > bad_extensions = { > accdb = 1, > accdr = 1, > ace = 1000, > ade = 1, > adp = 1, > ani = 1, > app = 1, > arj = 1, > asd = 1, > asf = 1, > asx = 1, > b64 = 1, > bas = 1, > bat = 1000, > > tag your bad extensions with a high multipicator (1000) > Well, it doesn't really work for me with mimetypes.conf. I have these entries in mimetypes.conf bad_extensions = { sh = 1000; } and in multimap.conf ATTACHMENT_BLACKLISTED { type = "filename"; filter = "extension"; map = "$CONFDIR/local.d/mimetype.conf"; action = "reject"; message = "A restricted file type was found"; skip_archives = true; } System reload required. If I now send a bash file with .sh to myself externally, it will not be rejected, but will end up in my junk folder as SPAM. 2024-04-16T08:39:54.059848+02:00 mx postfix/cleanup[2569186]: 0D3B4120093: message-id= 2024-04-16T08:39:54.270196+02:00 mx postfix/cleanup[2569186]: 0D3B4120093: milter-reject: END-OF-MESSAGE from mail-lj1-x22f.google.com[2a00:1450:4864:20 ::22f]: 4.7.1 Try again later; from= to= proto=ESMTP helo= Florian?s solution works better. Which is why I wanted to know why mimetypes.conf at all? So in principle Florian's solution meets the desired requirements. Best regards Andreas From m.stenzel at mail.xy-space.de Tue Apr 16 10:47:37 2024 From: m.stenzel at mail.xy-space.de (Martin Stenzel) Date: Tue, 16 Apr 2024 12:47:37 +0200 Subject: [Rspamd-Users] SPAM alerts with wrong action value Message-ID: <1e55-661e5780-db-7d20eb80@158785065> Hi list, I use the latest version of rspamd on a Linux server I set up rspamd according to the official documentation. >From time to time I receive mails which are tagged as spam (although they are NOT spam). The headers show this: ?X-Spam-Level: ****** X-Spamd-Result: default: False [0.59 / 12.00]; BAYES_HAM(-3.00)[100.00%]; FROM_EXCESS_QP(1.20)[]; REPLYTO_EXCESS_QP(1.20)[]; URI_COUNT_ODD(1.00)[47]; FORGED_SENDER(0.30)[info at popularresistance.org,bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_LIST_UNSUB(-0.01)[]; REDIRECTOR_FALSE(0.00)[facebook.com->list-manage.com:list-manage.com,popularresistance.org->list-manage.com:list-manage.com]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; NEURAL_SPAM(0.00)[0.874]; REDIRECTOR_URL(0.00)[list-manage.com,twitter.com]; FROM_HAS_DN(0.00)[]; FROM_NEQ_ENVFROM(0.00)[info at popularresistance.org,bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net]; PREVIOUSLY_DELIVERED(0.00)[m.stenzel at space.de]; RCVD_COUNT_FIVE(0.00)[6]; RCVD_TLS_LAST(0.00)[]; HAS_REPLYTO(0.00)[info at popularresistance.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; REPLYTO_EQ_FROM(0.00)[] X-Rspamd-Action: no action X-Rspamd-Queue-Id: B3BDB6C02F8 X-Rspamd-Server: terve.xy-space.de There is the line?X-Rspamd-Action: no action But rspamd does act. My actions.conf looks like this: # local.d/actions.conf reject = 12.00; greylist = 10.00; rewrite_subject = 6.00; add_header = 5.52; subject = "***** SPAM ALERT ***** %s" The subject of the header looks like this:***** SPAM ALERT ***** Popular Resistance Daily Digest In the logs I find this: ? (rspamd_proxy) ; proxy; rspamd_task_write_log: id: <33602bebba8fb7dd6e71fb413.67321fa2dd.20240416095943.c47741786b.2323a984 at mail17.suw91.mcdlv.net>, qid: , ip: 10.4.0.1, from: , (default: F (no action): [2.59/12.00] [BAYES_HAM(-3.00){100.00%;},FORGED_RECIPIENTS(2.00){m:m.stenzel at space.de;s:archive at mailarchive.xy-space.de;},FROM_EXCESS_QP(1.20){},REPLYTO_EXCESS_QP(1.20){},URI_COUNT_ODD(1.00){47;},FORGED_SENDER(0.30){info at popularresistance.org;bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net;},MIME_GOOD(-0.10){multipart/alternative;text/plain;},HAS_LIST_UNSUB(-0.01){},ARC_NA(0.00){},FROM_HAS_DN(0.00){},FROM_NEQ_ENVFROM(0.00){info at popularresistance.org;bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net;},HAS_REPLYTO(0.00){info at popularresistance.org;},MIME_TRACE(0.00){0:+;1:+;2:~;},NEURAL_SPAM(0.00){0.874;},PREVIOUSLY_DELIVERED(0.00){m.stenzel at space.de;},RCPT_COUNT_ONE(0.00){ 1;},RCVD_COUNT_FIVE(0.00){6;},RCVD_TLS_LAST(0.00){},REDIRECTOR_FALSE(0.00){facebook.com->list-manage.com:list-manage.com;popularresistance.org->list-manage.com:list-manage.com;},REDIRECTOR_URL(0.00){list-manage.com;twitter.com;},REPLYTO_EQ_FROM(0.00){},TO_DN_NONE(0.00){}]), len: 88228, time: 266.688ms, dns req: 67, digest: , rcpts: , mime_rcpts: ? -- Martin Stenzel ? er/ihm ? he/him m.stenzel at mail.xy-space.de An der Drehscheibe 9 D-50733 K?ln ? Cologne Deutschland ? Germany This message was checked by ESET Endpoint Antivirus for Linux. Detection Engine Version: 29070 (20240416). -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5674 bytes Desc: S/MIME Cryptographic Signature URL: From natan at epf.pl Tue Apr 16 11:06:05 2024 From: natan at epf.pl (natan) Date: Tue, 16 Apr 2024 13:06:05 +0200 Subject: [Rspamd-Users] mysql and whitelist Message-ID: <5bf67b22-03d5-4ead-838a-79f95f9d8d71@epf.pl> Hi Are there any plans to support individual white and black lists in MySQL? Or a "module" for integration with amavis? Why am I asking - because I use Spamassasin and Amavis and users have white and black lists in MySQL and it works. -- From t.hendricks at interpool.de Tue Apr 16 11:36:48 2024 From: t.hendricks at interpool.de (Tino Hendricks) Date: Tue, 16 Apr 2024 13:36:48 +0200 Subject: [Rspamd-Users] SPAM alerts with wrong action value In-Reply-To: <1e55-661e5780-db-7d20eb80@158785065> References: <1e55-661e5780-db-7d20eb80@158785065> Message-ID: Hi Martin, is it possible there?s an additional spam actor in the queue who adds the header? > X-Spam-Level: ****** Or maybe the header is already in the incoming mail (which is/was sometimes the case with web.de , marking their own outgoing mail as SPAM). ????? > Am 16.04.2024 um 12:47 schrieb Martin Stenzel via Users : > > > Hi list, I use the latest version of rspamd on a Linux server > > I set up rspamd according to the official documentation. > > From time to time I receive mails which are tagged as spam (although they are NOT spam). > > The headers show this: > X-Spam-Level: ****** > X-Spamd-Result: default: False [0.59 / 12.00]; > BAYES_HAM(-3.00)[100.00%]; > FROM_EXCESS_QP(1.20)[]; > REPLYTO_EXCESS_QP(1.20)[]; > URI_COUNT_ODD(1.00)[47]; > FORGED_SENDER(0.30)[info at popularresistance.org,bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net]; > MIME_GOOD(-0.10)[multipart/alternative,text/plain]; > HAS_LIST_UNSUB(-0.01)[]; > REDIRECTOR_FALSE(0.00)[facebook.com->list-manage.com:list-manage.com,popularresistance.org->list-manage.com:list-manage.com]; > ARC_NA(0.00)[]; > TO_DN_NONE(0.00)[]; > RCPT_COUNT_ONE(0.00)[1]; > MIME_TRACE(0.00)[0:+,1:+,2:~]; > NEURAL_SPAM(0.00)[0.874]; > REDIRECTOR_URL(0.00)[list-manage.com,twitter.com]; > FROM_HAS_DN(0.00)[]; > FROM_NEQ_ENVFROM(0.00)[info at popularresistance.org,bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net]; > PREVIOUSLY_DELIVERED(0.00)[m.stenzel at space.de]; > RCVD_COUNT_FIVE(0.00)[6]; > RCVD_TLS_LAST(0.00)[]; > HAS_REPLYTO(0.00)[info at popularresistance.org]; > TO_MATCH_ENVRCPT_ALL(0.00)[]; > REPLYTO_EQ_FROM(0.00)[] > X-Rspamd-Action: no action > X-Rspamd-Queue-Id: B3BDB6C02F8 > X-Rspamd-Server: terve.xy-space.de > There is the line X-Rspamd-Action: no action > > > But rspamd does act. > > My actions.conf looks like this: > # local.d/actions.conf > reject = 12.00; > greylist = 10.00; > rewrite_subject = 6.00; > add_header = 5.52; > subject = "***** SPAM ALERT ***** %s" > > The subject of the header looks like this:***** SPAM ALERT ***** Popular Resistance Daily Digest > > > In the logs I find this: > > > (rspamd_proxy) ; proxy; rspamd_task_write_log: id: <33602bebba8fb7dd6e71fb413.67321fa2dd.20240416095943.c47741786b.2323a984 at mail17.suw91.mcdlv.net>, qid: , ip: 10.4.0.1, from: , (default: F (no action): [2.59/12.00] [BAYES_HAM(-3.00){100.00%;},FORGED_RECIPIENTS(2.00){m:m.stenzel at space.de;s:archive at mailarchive.xy-space.de;},FROM_EXCESS_QP(1.20){},REPLYTO_EXCESS_QP(1.20){},URI_COUNT_ODD(1.00){47;},FORGED_SENDER(0.30){info at popularresistance.org;bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net;},MIME_GOOD(-0.10){multipart/alternative;text/plain;},HAS_LIST_UNSUB(-0.01){},ARC_NA(0.00){},FROM_HAS_DN(0.00){},FROM_NEQ_ENVFROM(0.00){info at popularresistance.org;bounce-mc.us2_5197842.1426441-67321fa2dd at mail17.suw91.mcdlv.net;},HAS_REPLYTO(0.00){info at popularresistance.org;},MIME_TRACE(0.00){0:+;1:+;2:~;},NEURAL_SPAM(0.00){0.874;},PREVIOUSLY_DELIVERED(0.00){m.stenzel at space.de;},RCPT_COUNT_ONE(0.00){ > 1;},RCVD_COUNT_FIVE(0.00){6;},RCVD_TLS_LAST(0.00){},REDIRECTOR_FALSE(0.00){facebook.com->list-manage.com:list-manage.com;popularresistance.org->list-manage.com:list-manage.com;},REDIRECTOR_URL(0.00){list-manage.com;twitter.com;},REPLYTO_EQ_FROM(0.00){},TO_DN_NONE(0.00){}]), len: 88228, time: 266.688ms, dns req: 67, digest: , rcpts: , mime_rcpts: > > > > > > > > > > > > > -- > Martin Stenzel ? er/ihm ? he/him > m.stenzel at mail.xy-space.de > > An der Drehscheibe 9 > D-50733 K?ln ? Cologne > Deutschland ? Germany > > > > This message was checked by ESET Endpoint Antivirus for Linux. > Detection Engine Version: 29070 (20240416). > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users From ray at rodonnell.ie Tue Apr 16 13:55:01 2024 From: ray at rodonnell.ie (Ray O'Donnell) Date: Tue, 16 Apr 2024 14:55:01 +0100 Subject: [Rspamd-Users] mysql and whitelist In-Reply-To: <5bf67b22-03d5-4ead-838a-79f95f9d8d71@epf.pl> References: <5bf67b22-03d5-4ead-838a-79f95f9d8d71@epf.pl> Message-ID: <15194f2b-cb17-414d-b158-0c0064cc5451@rodonnell.ie> On 16/04/2024 12:06, natan wrote: > Hi > Are there any plans to support individual white and black lists in MySQL? > Or a "module" for integration with amavis? > Why am I asking - because I use Spamassasin and Amavis and users have > white and black lists in MySQL and it works. I'd be interested in this too, though using PostgreSQL. Ray. -- Raymond O'Donnell // Galway // Ireland ray at rodonnell.ie From jose.celestino at gmail.com Tue Apr 16 18:16:42 2024 From: jose.celestino at gmail.com (jose.celestino at gmail.com) Date: Tue, 16 Apr 2024 19:16:42 +0100 Subject: [Rspamd-Users] mysql and whitelist In-Reply-To: <5bf67b22-03d5-4ead-838a-79f95f9d8d71@epf.pl> References: <5bf67b22-03d5-4ead-838a-79f95f9d8d71@epf.pl> Message-ID: Hi, We had the same need and ended up developing a system that feeds the SQL whitelists into the redis install. It works in near realtime and has the advantage that we keep the DB schema tunned to our best needs (integration with backoffices, other filters, etc). Comes with disadvantages too, but we're pleased. A ter?a, 16/04/2024, 12:41, natan escreveu: > Hi > Are there any plans to support individual white and black lists in MySQL? > Or a "module" for integration with amavis? > Why am I asking - because I use Spamassasin and Amavis and users have > white and black lists in MySQL and it works. > -- > -- > Users mailing list > Users at lists.rspamd.com > https://lists.rspamd.com/mailman/listinfo/users > From natan at epf.pl Wed Apr 17 11:16:36 2024 From: natan at epf.pl (natan) Date: Wed, 17 Apr 2024 13:16:36 +0200 Subject: [Rspamd-Users] mysql and whitelist In-Reply-To: References: <5bf67b22-03d5-4ead-838a-79f95f9d8d71@epf.pl> Message-ID: Hi Is any one tested that solutions ? https://manpages.debian.org/testing/amavisd-new/Amavis::SpamControl::RspamdClient.3pm.en.html W dniu 16.04.2024 o?20:16, jose.celestino at gmail.com pisze: > Hi, > > We had the same need and ended up developing a system that feeds the SQL > whitelists into the redis install. > > It works in near realtime and has the advantage that we keep the DB schema > tunned to our best needs (integration with backoffices, other filters, > etc). Comes with disadvantages too, but we're pleased. > > > A ter?a, 16/04/2024, 12:41, natan escreveu: > >> Hi >> Are there any plans to support individual white and black lists in MySQL? >> Or a "module" for integration with amavis? >> Why am I asking - because I use Spamassasin and Amavis and users have >> white and black lists in MySQL and it works. >> -- >> -- >> Users mailing list >> Users at lists.rspamd.com >> https://lists.rspamd.com/mailman/listinfo/users >> -- From trashcan at ellael.org Fri Apr 19 17:17:40 2024 From: trashcan at ellael.org (Michael Grimm) Date: Fri, 19 Apr 2024 19:17:40 +0200 Subject: [Rspamd-Users] HOWTO: get module fuzzy_check running Message-ID: Hi, this is rspamd 3.8.4, redis 7.2.4, dovecot 2.3.21, and postfix 3.9.0 (milter) running in a jail with FreeBSD 14-STABLE as host. This setup has been running for years now, but only recently ;-) I noticed that I am blocked regarding fuzzy_check: FUZZY_BLOCKED(0.00)[rspamd.com]; Both of my servers are privately used by a handful users and show very low amounts of mail. Thus I do qualify for free usage of the feeds [1] But before contacting rspamd.com as advertised in [1], I would like to understand, if my setup of rspamd isn't the cause for becoming blocked sometimes during the last years. I do run rspamd with only a few modifications in 'rspamd/local.d': #) arc and dkim_signing #) logging.inc (syslog, info level, enable debugging for fuzzy_check module) #) milter_headers.conf #) multimap.conf (whitelisting internal IPs and some sending 'from' headers) #) options.inc (local_addrs and control_socket) #) redis.conf (servers as socket) #) worker-controller.inc (bind_socket and secure_ip) #) worker-normal.inc (disabled) #) worker-proxy.inc (milter, bind_socket, self-scan mode) and #) fuzzy_check.conf (whitelist local IPs, nothing else) Here are my questions I cannot answer after reading the documentation and searching the internet: #) What does 'servers = "round-robin:fuzzy1.rspamd.com:11335,fuzzy2.rspamd.com:11335";' [2] in 'rspamd/modules/fuzzy_check.conf mean: +) Is my rspamd instance checking hashes of a given mail at these servers using UDP at remote port 11335? or +) Is my rspamd instance using these servers, protocol and ports to download hashes on a regular basis? #) If I do *not* want to teach fuzzy storage, I do not need to touch the default worker-fuzzy.inc? #) Does one need to configure the lua module fuzzy_collect, and if so, how? rspamd[43350]: <3j8qks>; cfg; rspamd_config_is_module_enabled: lua module fuzzy_collect is enabled but has not been configured Thanks in advance and regards, Michael [1] https://rspamd.com/doc/usage_policy.html [2] both servers do use the very same IP address. Is this on purpose? From trashcan at ellael.org Sat Apr 20 19:23:51 2024 From: trashcan at ellael.org (Michael Grimm) Date: Sat, 20 Apr 2024 21:23:51 +0200 Subject: [Rspamd-Users] Map of IPs in CIDR notation but with exceptions? Message-ID: Hi, I wonder if there is a way to define a map of IPs like: All 10.1.1.0/24 but not 10.1.1.222? Tests like ? 10.1.1.0/24,!10.1.1.222 10.1.1.0/24,-10.1.1.222 ? failed with "invalid IP address: !10.1.1.222" and "invalid IP address: -10.1.1.222" Is there a way to achieve this? Thanks and regards, Michael From moiseev at mezonplus.ru Sat Apr 20 21:12:51 2024 From: moiseev at mezonplus.ru (Alexander Moisseev) Date: Sun, 21 Apr 2024 00:12:51 +0300 Subject: [Rspamd-Users] Map of IPs in CIDR notation but with exceptions? In-Reply-To: References: Message-ID: <9aa4db5e-1744-e011-781b-38872876d5d3@mezonplus.ru> On 20.04.2024 22:23, Michael Grimm via Users wrote: > Hi, > > I wonder if there is a way to define a map of IPs like: > > All 10.1.1.0/24 but not 10.1.1.222? > > Tests like ? > > 10.1.1.0/24,!10.1.1.222 > 10.1.1.0/24,-10.1.1.222 > > ? failed with "invalid IP address: !10.1.1.222" and "invalid IP address: -10.1.1.222" > > Is there a way to achieve this? > You can use conditional maps: https://rspamd.com/doc/modules/multimap.html#conditional-maps Example: https://github.com/moisseev/rspamd-multimap-bl/blob/9608a091b9f4ad9f353ee8f061ea0fd43e6685ba/local.d/multimap.conf#L9 From trashcan at ellael.org Sun Apr 21 09:21:15 2024 From: trashcan at ellael.org (Michael Grimm) Date: Sun, 21 Apr 2024 11:21:15 +0200 Subject: [Rspamd-Users] Map of IPs in CIDR notation but with exceptions? In-Reply-To: <9aa4db5e-1744-e011-781b-38872876d5d3@mezonplus.ru> References: <9aa4db5e-1744-e011-781b-38872876d5d3@mezonplus.ru> Message-ID: Alexander Moisseev via Users wrote: > > On 20.04.2024 22:23, Michael Grimm via Users wrote: >> I wonder if there is a way to define a map of IPs like: >> All 10.1.1.0/24 but not 10.1.1.222? >> Tests like ? >> 10.1.1.0/24,!10.1.1.222 >> 10.1.1.0/24,-10.1.1.222 >> ? failed with "invalid IP address: !10.1.1.222" and "invalid IP address: -10.1.1.222" >> Is there a way to achieve this? > > You can use conditional maps: https://rspamd.com/doc/modules/multimap.html#conditional-maps > Example: https://github.com/moisseev/rspamd-multimap-bl/blob/9608a091b9f4ad9f353ee8f061ea0fd43e6685ba/local.d/multimap.conf#L9 Thanks, that was what I was looking for and didn't find ;-) Regards, Michael From usenet at schani.com Sun Apr 21 20:27:45 2024 From: usenet at schani.com (christian) Date: Sun, 21 Apr 2024 22:27:45 +0200 Subject: [Rspamd-Users] Clear Spam Mail Score 33 goes into Greylist? Message-ID: <0109919a-4f95-4671-8787-232ae3dd0d75@schani.com> Hello, the proportion of emails that are greylisted is over 30%. Emails with a score greater than 30 will still be greylisted. my setup: greylist = 1 add header = 4 rewrite subject = 15 reject = 70 I thought that only a transitional range between 1 and 4 would be greylisted, instead even scores with 48 would be greylisted. - The Most Powerful Keto Gummies-soft reject-greylist 48.65/70 - The death of black coffee-soft reject-greylist 47.33 / 70 - Harvard Prostate Study: Do You See This In Your Water? soft reject greylist 30.70 / 70 Actually, emails with a score < 70 should be kept. Instead, they are blocked. Why does an email with score 25 go into Greylist when rewrite subject is actually specified? Can i specify an upper limit for Greylist? Thank you very much for a tip Christian 33.95 / 70 Sort by: BAD_TLD (10) [gipolsan.pics] BAYES_SPAM (8) [100.00%] MAILBABY_CH_ATTACHMENT_DIGEST_IN_SPAM (7) [548eae78747a4546] IP_REPUTATION_SPAM (2.65388) [asn: 35592(0.06), country: CZ(0.00), ip: 89.187.144.170(0.38)] FORGED_RECIPIENTS (2) [m:karl-heinz.fgds at fgdsf.de,s:info at gfggdsfg.de] CONTENT_BLACKLISTED (2) BAD_REP_POLICIES (2) SPF_REPUTATION_SPAM (1.174702) [0.58735081712802] RCVD_VIA_SMTP_AUTH (-1) FROM_EQ_ENVFROM (-1) MID_RHS_NOT_FQDN (0.5) GENERIC_REPUTATION (0.426952) [0.42695222990883] ONCE_RECEIVED (0.1) MIME_GOOD (-0.1) [multipart/related,multipart/alternative,text/plain] RCVD_NO_TLS_LAST (0.1) MISSING_XM_UA (0.1) MX_GOOD (-0.01) [] FROM_HAS_DN (0) RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER (0) [191.96.208.174:received] DNSWL_BLOCKED (0) [191.96.208.174:received] GREYLIST (0) [greylisted,Sun, 21 Apr 2024 19:51:28 GMT,new record] From trashcan at ellael.org Mon Apr 22 19:42:49 2024 From: trashcan at ellael.org (Michael Grimm) Date: Mon, 22 Apr 2024 21:42:49 +0200 Subject: [Rspamd-Users] Can't get cluster WebUI running Message-ID: <81DDE134-C0E5-40EB-B924-82F1A182A54F@ellael.org> Hi, I do have have two rspamd instances running on two distinct servers with two different nginx instances. At each server both nginx and rspamd run in distinct jails (FreeBSD). nginx at server1 (10.1.1.2) rspamd at server1 (10.1.1.1) nginx at server2 (10.2.2.2) rspamd at server2 (10.2.2.1) server1.tld: nginx (reverse proxy with TLS) server2.tld: nginx (reverse proxy with TLS) I do get the WebUI running for each server1 and server2 independently (server1 shown, only): nginx at server1.tld (https://rspamd.com/doc/faq.html#how-to-use-the-webui-behind-a-proxy-server) location /rspamd/ { proxy_pass http://10.1.1.1:11334/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For ""; } rspamd at server1.tld (https://rspamd.com/doc/configuration/options.html#neighbours-list) local.d/worker-controller.inc: password = "secret"; enable_password = "secret"; bind_socket = "/var/run/rspamd/rspamd.sock mode=0600"; # postfix milter communication bind_socket = "10.1.1.1:11334"; secure_ip = [ "10.1.1.1", "10.2.2.2" ]; This works fine. But whenever I do define identical neighbour settings at both rspamd instances like ... local.d/options.inc: neighbours { server1 { host = "https://server1.tld:443"; } server2 { host = "https://server2.tld:443"; } } ? I do get error messages from WebUI: "server1> Cannot get server status" "server2> Cannot get server status" "Request failed" "server1> Cannot receive stats data" "server2> Cannot receive stats data" "Request failed" Any hints are highly appreciated. Thanks and regards, Michael Grimm From trashcan at ellael.org Wed Apr 24 21:03:28 2024 From: trashcan at ellael.org (Michael Grimm) Date: Wed, 24 Apr 2024 23:03:28 +0200 Subject: [Rspamd-Users] Can't get cluster WebUI running In-Reply-To: <81DDE134-C0E5-40EB-B924-82F1A182A54F@ellael.org> References: <81DDE134-C0E5-40EB-B924-82F1A182A54F@ellael.org> Message-ID: <7B819FC7-C980-4EB1-8D9C-D65CADBBA974@ellael.org> Michael Grimm via Users wrote: > Hi, > > I do have have two rspamd instances running on two distinct servers with two different nginx instances. > > > At each server both nginx and rspamd run in distinct jails (FreeBSD). > > nginx at server1 (10.1.1.2) rspamd at server1 (10.1.1.1) > nginx at server2 (10.2.2.2) rspamd at server2 (10.2.2.1) > > server1.tld: nginx (reverse proxy with TLS) > server2.tld: nginx (reverse proxy with TLS) > > > I do get the WebUI running for each server1 and server2 independently (server1 shown, only): > > nginx at server1.tld (https://rspamd.com/doc/faq.html#how-to-use-the-webui-behind-a-proxy-server) > location /rspamd/ { > proxy_pass http://10.1.1.1:11334/; > proxy_set_header Host $host; > proxy_set_header X-Real-IP $remote_addr; > proxy_set_header X-Forwarded-For ""; > } > > rspamd at server1.tld (https://rspamd.com/doc/configuration/options.html#neighbours-list) > local.d/worker-controller.inc: > password = "secret"; > enable_password = "secret"; > bind_socket = "/var/run/rspamd/rspamd.sock mode=0600"; # postfix milter communication > bind_socket = "10.1.1.1:11334"; > secure_ip = [ "10.1.1.1", "10.2.2.2" ]; > > This works fine. > > But whenever I do define identical neighbour settings at both rspamd instances like ... > > local.d/options.inc: > neighbours { > server1 { host = "https://server1.tld:443"; } > server2 { host = "https://server2.tld:443"; } > } > > ? I do get error messages from WebUI: > > "server1> Cannot get server status" > "server2> Cannot get server status" > "Request failed" > "server1> Cannot receive stats data" > "server2> Cannot receive stats data" > "Request failed" Well, that was tough, but I finally solved it. 1) local.d/options: neighbours { server1 { host = "https://server1.tld:443"; path = "/rspamd/"; } server2 { host = "https://server2.tld:443"; path = "/rspamd/"; } } 2) nginx configuration: location /rspamd/ { proxy_pass http://10.1.1.1:11334/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For ""; add_header Access-Control-Allow-Credentials 'true'; # IMPORTANT! } FTR: 1) I couldn't find any hint about "add_header Access-Control-Allow-Credentials 'true'" in the rspamd documents. Hope that's save to set? 2) https://rspamd.com/doc/configuration/options.html#neighbours-list taught me that it might be nessessary to separate 'host' and 'path' in the neighbour list. BUT: "server1 { host = "https://server1.tld/rspamd/:443"; }" didn't work for me, dunno why. Do you? Regards, Michael