[Rspamd-Users] How to handle MIME encoded headers?

[Dan Swartzendruber]

On 2024-06-17 12:20, G.W. Haywood wrote:
> Hi there,
> 
> On Mon, 17 Jun 2024, Dan Swartzendruber via Users wrote:
> 
>> Been using rspamd for a couple of weeks now, and it works just fine.  
>> The only issue I'm having is somehow getting tons of financial 
>> clickbait articles that score low on all defaults, so every day I have 
>> to delete dozens of these.  They get sneaky and encode the subject 
>> lines so instead of seeing things like 'subject: Taiwan in Dаnger Amіd 
>> Chinese Drіlls' it is 'subject: 
>> =?UTF-8?B?VGFpd2FuIGluIETQsG5nZXIgQW3RlmQgQ2hpbmVzZSBEctGWbGxz?='.  So 
>> the normal header examination setup in multimap.conf won't work 
>> because the headers have been decoded.  I get not authentic emails 
>> with the subjects encoded this way so I'd like to flag these as spam, 
>> but not high enough to be outright rejected.  So I'd like to look at 
>> the undecoded subject headers and if I see a regex like 
>> '=\?UTF-8\?.*\?=' it would add 7.0 to the score. Except as said, the 
>> headers are decoded.  It seems like the following would work (if the 
>> subject header was undecoded):
>> 
>> mime_subject_spam {
>>        type = "header"; <=== needs changing?
>>        header = "subject";
>>        filter = "regexp:/.*UTF\-8\?.*\?=/i";
>>        map = "/var/rspamd/maps/mime_subject_spam.map"; <=== don't need 
>> a map but it complains, so an empty file?
>>        symbol = "MIME_SUBJECT_SPAM";
>>        description = "Detect mime-encoded spam subjects";
>>        score = 7.0;
>>        regexp = true;
>> }
> 
> Have you tried using the /u flag in your regexes?

I'm not clear on what the /u flag expects.  I tried something like:

mime_subject_spam {
         type = "header";
         header = "subject";
         filter = "regexp:/(^|.*)=\?UTF\-8@/iu";
         map = "/var/rspamd/maps/mime_subject_spam.map";
         symbol = "MIME_SUBJECT_SPAM";
         description = "Detect mime-encoded spam subjects";
         score = 7.0;
         regexp = true;
}

thinking for regexp strings, '?' and '-' need to be escaped.  Yet, in 
configdump output:

     mime_subject_spam {
         filter = "regexp:/(^|.*)=?UTF-8@/iu";
         regexp = true;
         score = 7;
         symbol = "MIME_SUBJECT_SPAM";
         type = "header";
         description = "Detect mime-encoded spam subjects";
         header = "subject";
         map = "/var/rspamd/maps/mime_subject_spam.map";

The '\' are gone?  Is that cosmetic, or expected?  In any event, the 
above didn't work. Also without a map file, this config is invalid, so I 
gave it an empty file which is probably wrong :)

> https://rspamd.com/doc/modules/regexp.html
> 
>> p.s. I have to say the docs for rspamd are extremely complete, except 
>> that everything I look at explains what X and Y and Z are, but I don't 
>> see examples of how to do what I would like (FWIW, I ginned up the 
>> config here by looking at samples in modules.d, but that isn't exactly 
>> what I need.)  What am I missing here?  Any tips appreciated :)
> 
> It isn't easy to find your way around the docs I'm afraid.
> 
> There's an alternative take on them which *might* help
> 
> https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-introduction/
> 
> but not necessarily with this specific question.
> 
> --
> 
> 73,
> Ged.