[Rspamd-Users] Intermittent Unit Test Failures

[Steve Sturges (ststurge)]

I’m running rspamd 3.1 within an Alpine 3.15 container (Lua 5.1/LuaJIT 2.1) and have seen some intermittent failures with the
rspamd lua unit tests, in particular in lua/unit/lua_util.extract_specific_urls.lua.

Any ideas what could be causing this - it seems order of URLs in a lua table is not predictable?
I have not yet tried with rspamd 3.2.

From /usr/test within my container,

rspamd-test -p /rspamd/lua --verbose --debug-log

Across multiple runs, I see different results (output trimmed for simplicity):

Run 1:
641 tests 635 passed 809425 assertions 2 failed 0 errors 4 unassertive 0 pending

extract_specific_urls, backward compatibility case #3:
Failed asserting that
  (actual)   : {[1] = domain3.org<http://domain3.org>, [2] = test.com<http://test.com>}
 equals to
  (expected) : {[1] = domain4.co.net<http://domain4.co.net>, [2] = test.com<http://test.com>}
…

Run 2:
641 tests 637 passed 809517 assertions 0 failed 0 errors 4 unassertive 0 pending

Run 3:
641 tests 635 passed 809425 assertions 2 failed 0 errors 4 unassertive 0 pending

extract_specific_urls, backward compatibility case #3:
Failed asserting that
  (actual)   : {[1] = domain4.co.net<http://domain4.co.net>, [2] = domain3.org<http://domain3.org>}
 equals to
  (expected) : {[1] = domain4.co.net<http://domain4.co.net>, [2] = test.com<http://test.com>}
…

Run 4:
641 tests 635 passed 809483 assertions 2 failed 0 errors 4 unassertive 0 pending

extract_specific_urls, backward compatibility case #3:
Failed asserting that
  (actual)   : {[1] = test.com<http://test.com>, [2] = domain3.org<http://domain3.org>}
 equals to
  (expected) : {[1] = domain4.co.net<http://domain4.co.net>, [2] = test.com<http://test.com>}
…

Run 5:
641 tests 635 passed 809431 assertions 2 failed 0 errors 4 unassertive 0 pending

extract_specific_urls, backward compatibility case #3:
Failed asserting that
  (actual)   : {[1] = domain3.org<http://domain3.org>, [2] = domain4.co.net<http://domain4.co.net>}
 equals to
  (expected) : {[1] = domain4.co.net<http://domain4.co.net>, [2] = test.com<http://test.com>}
…

Run 6:
641 tests 637 passed 809517 assertions 0 failed 0 errors 4 unassertive 0 pending