[Rspamd-Users] redis OOM

[Wiethoff, Helge]

Hi Alexander,

> I would check redis-cli `info memory` output and rspamd log.

----------------
host:/etc/rspamd# redis-cli info memory
# Memory
used_memory:2577962576
used_memory_human:2.40G
used_memory_rss:2637262848
used_memory_rss_human:2.46G
used_memory_peak:2579614632
used_memory_peak_human:2.40G
total_system_memory:8201797632
total_system_memory_human:7.64G
used_memory_lua:1264640
used_memory_lua_human:1.21M
maxmemory:3221225472
maxmemory_human:3.00G
maxmemory_policy:volatile-ttl
mem_fragmentation_ratio:1.02
mem_allocator:jemalloc-3.6.0
----------------

The rspamd log "looks kind of normal" beside this repeating message:
2019-06-28 09:33:40 #3067(rspamd_proxy) <9cb4a3>; lua; neural.lua:487: cannot learn ANN tRFANNCA81FF5E85529498260: too many ham samples: 320

When redis is oom the log is a bit different of course:
----------------
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; proxy; spf_check_list: use cached record for xxx (0x52554cd82da671eb) in LRU cache for 300 seconds, 20/2000 elements in the cache
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; proxy; make_dns_request_task_common: <105302532.10176.1561635628677.JavaMail.cfmx at 127.0.0.1> stop resolving on reaching 64 requests
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; lua; clamav.lua:132: clamav: message or mime_part is clean
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; lua; common.lua:183: failed to save virus cache for OK -> "rs_clamav_9bd97a6f88e3e4521d5b7c02eaba09df1a35de9b78b7bb41e7f2798fe69db1267e8ea04c95d67b1706e42b92f9876d7cfc3f40fc01ce32a869f0904c9dd6cc0d": OOM command not allowed when used memory > 'maxmemory'.
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; lua; ip_score.lua:123: got error while IP score changing: OOM command not allowed when used memory > 'maxmemory'.
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; lua; greylist.lua:396: greylisted until "Thu, 27 Jun 2019 14:46:04 GMT", new record
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; proxy; rspamd_add_passthrough_result: <105302532.10176.1561635628677.JavaMail.cfmx at 127.0.0.1>: set pre-result to 'soft reject' (no score): 'Try again later' from greylist(1)
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; lua; greylist.lua:355: got error OOM command not allowed when used memory > 'maxmemory'. when setting greylisting record on server 127.0.0.1
2019-06-27 16:41:04 #5391(rspamd_proxy) <81e6bf>; lua; neural.lua:487: cannot learn ANN tRFANNCA81FF5E85529498260: too many spam samples: 1101
----------------

Beside that: I was trying to limit the Bayes entries but local.d/statistic.conf conflicts with local.d/classifier-bayes.conf:
new_schema = true;
expire = 8640000;

Maybe it's a good idea to create a redis instance for bayes entries?

-Helge