[Rspamd-Users] Redis huge database

[Alexander Moisseev]

On 23.11.2022 10:31, azurit at pobox.sk wrote:
> 
> Citát Alexander Moisseev via Users <users at lists.rspamd.com>:
> 
>> On 22.11.2022 22:45, azurit at pobox.sk wrote:
>>> i'm having problems with Redis database - it's huge and getting bigger, no matter what i do. Redis is taking more and more memory. If i look at the keys i see about 500 000 keys with name similar to 'RS_1028729928929298385'. Is this normal?
>>>
>>
>> AFAIK no one has ever researched the correlation of number of statistical tokens and quality of classification, but I guess 0.5M keys may not be enough. Just for reference, I have about 4.2M RS_* keys in the bayes database (per user classifier is no enabled), used_memory_human:554.79M.
> 
> 
> My Redis database has almost 1 GB and Redis needs 4 GB of memory. With lower values, i'm getting this error from rspamd:
> Nov 17 00:13:56 server00 rspamd[4086]: <177d52>; lua; history_redis.lua:132: got error OOM command not allowed when used memory > 'maxmemory'. when writing history row: no value
> 
The on-disk .rdb file is compressed, so 1GB is a relatively large database.

> rspamd is the only service using this Redis instance.
> 
As you store everything related to Rspamd in the single Redis instance there is no easy way to determine how much database space each module consumes, I'm afraid. Probably you need to to count the number of keys matching patterns with redis-cli.
Also some excessive numbers (like stored fuzzy hashes, history nrows, etc.) can indirectly indicate the source of the problem.