[Rspamd-Users] Scan time

[Srikrishnan Chitoor]

 Hi:
 I have clearly mentioned that we are testing. Let me clarify further:
 1. It has one new domain with 1 email ID. Domain MX pointing to this server,
 2. It is not MX for any other system. 3. It processed all of 1 email (which was the test email we sent) of size 2Kb. 4. The slow log said what are the processes taking time (I has posted the logs).

 Processing time was 4 secs. When we disabled Fuzzy checks, processing time came down to 0.9 seconds.
 In fact, am offering to host a mirror of the RSPAMD lists for bringing down latency to users in this region. 

 I like rspamd, its concept and would like to help in anyway we can. 

 As to why we think the scan times are high:
1. We run a live system with qpsmtpd and scan times are much lesser (typically at around 1-2 secs), on     a system that receives around 100,000 non SPAM emails and rejects about 5 times more    emails.
2. Going thru https://rspamd.com/performance.html, we wanted to see if rspamd    can outdo qpsmtpd.3. If, even for a single mail (that too a test email we sent), if scan time is 4 secs, how can we be    confident of it scaling?
If you need more data, I will be willing to share. I posted the technical information so that others can see it. I do not need any reply though.

    On Wednesday, June 29, 2022 at 07:11:50 PM GMT+5:30, G.W. Haywood via Users <users at lists.rspamd.com> wrote:  
 
 Hi there,

On Wed, 29 Jun 2022, Srikrishnan Chitoor via Users wrote:

> Finally identified the problem. The server is in India and there
> seems to be high latency for DNS tests that involve RSPAMD lists,
> especially the fuzzy check.  I disabled the fuzzy check module and
> the time for scanning came down from average of 4.5 secs to 0.9
> seconds.

As I said in my reply to your earlier post:

>> If you think there's a problem, you need to say why you think that.

Yes, scanning time is increased if you do more work in the scanning.
That should be no surprise to anyone.  It is not a "problem".

You have not yet explained your problem.  For example, it might be
that you are trying to process thousands of messages per second but
that with the system you have implemented you can only process
hundreds of messages per second.  That would qualify as a problem.
The timeout on mail transfers is typically a few days.  That a mail
message might take 3.6 seconds longer to reach its destination than
would be the case if it weren't scanned is not what I would consider
to be a problem.

Please explain why you maintain that the scanning time is a problem
for you.  In case it isn't clear, multiple messages will (usually) be
processed in parallel, so you cannot calculate the maximum throughput
of your mail system simply by taking the reciprocal of the average
message processing time.

If scanning IS creating a bottleneck you need to demonstrate it by
providing some evidence.

> Is there any provision to mirror the RSPAMD lists locally? We can
> help with the Hardware if required.

This appears to me to be an example of an XY problem:

https://en.wikipedia.org/wiki/XY_problem

It seem to me that you have made some assumptions.  Please explain
what they are, why you have made them, and why you believe that a
local mirror might help you.  Until you do that I can't help more.

-- 

73,
Ged.
-- 
Users mailing list
Users at lists.rspamd.com
https://lists.rspamd.com/mailman/listinfo/users