[Rspamd-Users] Problems with multimap (which I'm sure used to work...)

[Tim Cutts]

Dear list,

I hope you can help me debug a multimap rule I’m having trouble with.

I get a lot of spam from a particular source, and their common feature is the all have received headers like this one:

Received: from mail.understandingtheimpacts.com <http://mail.understandingtheimpacts.com/> (vpsnode12.webstudio26.com <http://vpsnode12.webstudio26.com/> [185.250.243.24]) by cyclin.thecutts.org <http://cyclin.thecutts.org/> (Postfix) with ESMTP id 45064300019 for <tim at thecutts.org <mailto:tim at thecutts.org>>; Thu, 17 Oct 2019 13:15:37 +0100 (BST)

The numbers in the vpsnode hostname vary, but the pattern is the same.

So, I added a rule in local.d/multimap.conf:

TJRC_RECEIVED_BLACKLIST {
           type = "received";
           filter = "real_hostname";
           description = "Found in Tim's Received blacklist";
           map = "/${LOCAL_CONFDIR}/local.d/received_blacklist.map";
           symbol = "TJRC_RECEIVED_BLACKLIST";
           regexp = true;
}

And the .map file looks like this:

/vpsnode\d+\.webstudio\d+\.com/i

A recent message I received with the above Received header did not trigger this rule, when I look at the headers rspamd added:

X-Spamd-Result: default: False [4.09 / 10.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[understandingtheimpacts.com <http://understandingtheimpacts.com/>:s=dkim]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[tim at thecutts.org <mailto:tim at thecutts.org>]; TO_DN_NONE(0.00)[]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; FORGED_SENDER_VERP_SRS(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; DKIM_TRACE(0.00)[understandingtheimpacts.com <http://understandingtheimpacts.com/>:+]; DMARC_POLICY_ALLOW(-0.50)[understandingtheimpacts.com <http://understandingtheimpacts.com/>,none]; SUBJECT_ENDS_EXCLAIM(1.00)[]; ENVFROM_VERP(0.00)[]; FORGED_SENDER(0.00)[clearone_debt_settlement at understandingtheimpacts.com <mailto:clearone_debt_settlement at understandingtheimpacts.com>,clearone_debt_settlement-tim=thecutts.org at understandingtheimpacts.com <mailto:clearone_debt_settlement-tim=thecutts.org at understandingtheimpacts.com>]; RCVD_NO_TLS_LAST(0.10)[]; SENDER_REP_SPAM(0.00)[asn: 43260(0.00), country: TR(0.01), ip: 185.250.243.24(0.00)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TJRC_SENDER_FROM_BLACKLIST(2.00)[clearone_debt_settlement-tim=thecutts.org <http://thecutts.org/>]; ASN(0.00)[asn:43260, ipnet:185.250.243.0/24, country:TR]; FROM_NEQ_ENVFROM(0.00)[clearone_debt_settlement at understandingtheimpacts.com <mailto:clearone_debt_settlement at understandingtheimpacts.com>,clearone_debt_settlement-tim=thecutts.org at understandingtheimpacts.com <mailto:clearone_debt_settlement-tim=thecutts.org at understandingtheimpacts.com>]; GREYLIST(0.00)[pass,body]; RCVD_COUNT_TWO(0.00)[2]

There’s no TJRC_RECEIVED_BLACKLIST there.

But if I pass the same message through rspamc symbols, it does trigger the rule:

Results for file: stdin (0.356 seconds)
[Metric: default]
Action: reject
Spam: true
Score: 18.59 / 10.00
…
Symbol: TJRC_RECEIVED_BLACKLIST (5.00)[vpsnode12.webstudio26.com <http://vpsnode12.webstudio26.com/>]
…

So the rule does appear to be configured more or less correctly.

The score’s much higher now because various blacklists have picked the new spam servers up, but the spammers change server every day so the RBLs are always playing catch-up and some of this spam still gets through, it’s only this DNS name pattern that stays constant.

What am I missing?  Why is the rule triggered by rspamc, but not when the original message came in?   It suggests that received header isn’t there yet at the time postfix passes the message to the rspamd milter.  Is that the issue?   This is rspamd 2.0 on Ubuntu 18.04, from the rspamd repository:

# apt-cache policy rspamd
rspamd:
 Installed: 2.0-1~bionic
 Candidate: 2.0-1~bionic
 Version table:
*** 2.0-1~bionic 500
       500 http://rspamd.com/apt-stable <http://rspamd.com/apt-stable> bionic/main amd64 Packages
       100 /var/lib/dpkg/status

Thanks in advance,

Tim