[Rspamd-Users] Things I don't understand yet

Tue Feb 13 12:19:31 UTC 2024

Hello again,

On Mon, 12 Feb 2024, christian via Users wrote:
> Am 12.02.2024 um 14:18 schrieb G.W. Haywood:
>> On Mon, 12 Feb 2024, christian via Users wrote:
>> 
>>> ... no evaluation is made based on my multimap ...
>> 
>> Here's something I found useful to try to make sense of things:
>> 
>> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-modules/
>> 
>> Just a guess, it might be an entry triggered in a whitelist or for
>> example mid, both of which you probably have enabled.
>> 
>> ... try ... rspamadm configtest
> 
> My rspamadm configtest gives the following:
>
> rspamadm configtest
> duplicate symbol: SUBJ_ALL_CAPS, skip registering
> unknown type of attribute enabled for regexp module
> cannot find dependency on symbol IS_IN_WHITELIST for symbol 
> FORCE_ACTION_MY_WHITELIST
> cannot find dependency on symbol FPROT_VIRUS for symbol 
> FORCE_ACTION_MY_WHITELIST
> syntax OK
>
> But unfortunately I can't do anything with the information.
> 
> What's more, I haven't changed anything in the symbols displayed.

With a configuration as complex as that of rspamd, in my view it's
absolutely essential to be very methodical when making any changes.

Do you have a record of all the changes that you made?  Do you have a
copy of your original configuration before you made any changes?  One
of the tools I use a great deal for this sort of work is 'diff'.  It
is, I suppose, really intended for people writing code, but it makes
it easy to compare huge sets of configuration options with changes in
files which hap-hazardly distributed within a large directory tree.

To test the latest installation instructions for rspamd which are
found on the rspamd Website, I followed them.  On a box which runs
Debian 'Bullseye', I ran the script below:

8<----------------------------------------------------------------------
#!/bin/bash
apt-get install -y lsb-release wget gpg
CODENAME=`lsb_release -c -s`
mkdir -p /etc/apt/keyrings
wget -O- https://rspamd.com/apt-stable/gpg.key | gpg --dearmor | tee /etc/apt/keyrings/rspamd.gpg > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main" | tee /etc/apt/sources.list.d/rspamd.list
echo "deb-src [signed-by=/etc/apt/keyrings/rspamd.gpg] http://rspamd.com/apt-stable/ $CODENAME main"  | tee -a /etc/apt/sources.list.d/rspamd.list
apt-get update
apt-get --no-install-recommends install rspamd
8<----------------------------------------------------------------------

It only took a minute or so, and ran to completion with no issues.

The installation script is more or less according to the instructions at

https://rspamd.com/downloads.html

but I removed 'sudo' because I ran the script as root.

Below is what happened with rspamd newly installed using the script:

8<----------------------------------------------------------------------
# rspamadm --version
Rspamadm 3.8.1
# rspamadm configtest
syntax OK
# ls -l /usr/share/rspamd/
total 264
-rw-r--r--  1 root root 242327 Jan 25 19:10 effective_tld_names.dat
drwxr-xr-x  2 root root   4096 Feb 13 10:45 elastic
drwxr-xr-x  2 root root   4096 Feb 13 10:45 languages
drwxr-xr-x 10 root root   4096 Feb 13 10:45 lualib
drwxr-xr-x  2 root root   4096 Feb 13 10:45 plugins
drwxr-xr-x  4 root root   4096 Feb 13 10:45 rules
drwxr-xr-x  6 root root   4096 Feb 13 10:45 www
# ls -l /etc/rspamd/
total 96
-rw-r--r-- 1 root root 1213 Jan 25 19:10 actions.conf
-rw-r--r-- 1 root root  365 Jan 25 19:10 cgp.inc
-rw-r--r-- 1 root root 1318 Jan 25 19:10 common.conf
-rw-r--r-- 1 root root 7614 Jan 25 19:10 composites.conf
-rw-r--r-- 1 root root 5154 Jan 25 19:10 groups.conf
-rw-r--r-- 1 root root  874 Jan 25 19:10 lang_detection.inc
drwxr-xr-x 2 root root 4096 Jan 25 19:31 local.d
-rw-r--r-- 1 root root 1186 Jan 25 19:10 logging.inc
drwxr-xr-x 2 root root 4096 Feb 13 10:45 maps.d
-rw-r--r-- 1 root root  921 Jan 25 19:10 metrics.conf
-rw-r--r-- 1 root root  703 Jan 25 19:10 modules.conf
drwxr-xr-x 2 root root 4096 Feb 13 10:45 modules.d
-rw-r--r-- 1 root root 2020 Jan 25 19:10 options.inc
drwxr-xr-x 2 root root 4096 Jan 25 19:31 override.d
-rw-r--r-- 1 root root 2787 Jan 25 19:10 rspamd.conf
drwxr-xr-x 2 root root 4096 Feb 13 10:45 scores.d
-rw-r--r-- 1 root root 1799 Jan 25 19:10 settings.conf
-rw-r--r-- 1 root root 2169 Jan 25 19:10 statistic.conf
-rw-r--r-- 1 root root  618 Jan 25 19:10 worker-controller.inc
-rw-r--r-- 1 root root  654 Jan 25 19:10 worker-fuzzy.inc
-rw-r--r-- 1 root root  525 Jan 25 19:10 worker-normal.inc
-rw-r--r-- 1 root root 1363 Jan 25 19:10 worker-proxy.inc
8<----------------------------------------------------------------------

As you can see the files all have a relatively recent timestamp, which
I find comforting when I do a new installtaion.  Directory timestamps
in this case are when they were created, that's not important here but
if I have trouble finding changes in more or less anything I'll often
look for files/directories in a directory tree with recent timestamps.

It's clear that you've done things with which the rspamd configuration
test is a little unhappy, but it _does_ say it's 'OK' so I think it
will probably be working as designed.  Whether or not it's working as
you intend is another matter.  You could always try the stricter test
according to the 'man' page - this is the result on my new config:

# rspamadm --var=DBDIR=/tmp configtest -c /etc/rspamd/rspamd.conf -s
syntax OK

If it were my system I think I'd want to be sure that all the warnings
were gone before I'd be happy.  Perhaps you could back out the changes
which you've made one-by-one until the output is silenced.  If you do
not have a record of the changes you could start with a fresh config.
Document each and every change you make, but also (1) test it and (2)
dump it to a file each time you change it, with something like

# rspamadm configdump > ~/rspamd.configdump.$(date -Iseconds)

so you always have a record of a sane configuration if you break it.
Unfortunately comparing the outputs of 'configdump' taken at different
times might not be very helpful; the order in which the various parts
are output seems to be non-deterministic.

To address the issue of some mails being passed through without being
processed by some of the modules I think maybe you should investigate
the parts of the configuration which implement whitelisting, but *not*
before you're happy that you know exactly what changes you have made
to the configuration and exactly what you expect these changes to do.

If you still can't figure out why you aren't getting the results you
expect, you can post to this list the changes which you have made (or
as I said earlier the full configuration dump).

If you have an example mail which was blocked by your configuration,
(and should have been) and one which you feel is similar but was not
(and should have been) perhaps you can put them somewhere where we
can see them and try to do some guesswork.

> If I understand correctly, Rspamd's basic settings are fetched from
> /usr/share/rspamd. I can then adjust this under /etc/rspamd/local.d.
> Some like multimap have no basic settings and are created themselves
> and read by rspamd.

Depending on how you look at things, fortunately or unfortunately the
paths for the configuration directories are themselves configurable.
That means the exact locations can be different in different systems
and the single biggest factors which determine the locations are the
way you installed rspamd and from where you downloaded the package.

If you installed as per the instruction on the rspamd Website your
understanding is correct.  I do not know what might happen if you
installed using packages from your Linux distribution, but if for
example you used the Debian package with just

apt-get install rspamd

there's a good chance that things will be four or five years out of
date and horribly broken.

> As a non-English speaker, I find it quite difficult to use the
> docs. Excuse me.

Your English is good but the documentation on the rspamd Website is I
fear not ideal for a new user.  That's why I pointed to an alternative
which, although it is somewhat out of date, tries a bit harder to make
the configuration make sense to someone new.  The array of options in
the rspamd configuration is immense and it's too easy to find yourself
lost in the woods.

-- 

73,
Ged.