[Rspamd-Users] csession; rspamd_controller_error_handler: http error occurred

[Peter Reinhold]

99.9% sure your admin panel is on the open internet, and the errors are 
because a crawler tried to retrieve robots.txt, which isn't there, and 
shouldn't be, because you should definitely not run the admin panel 
publicly.


---
Peter Reinhold


On 2024-01-03 19:06, Albrecht Backhaus wrote:
> *Von:/From:* Alexander Moisseev via Users <users at lists.rspamd.com>
> *Gesendet:/Sent:* Mittwoch, 03.01.2024 - 17:37
> *An:/To:* <users at lists.rspamd.com>
> *Kopie:/CC:* Alexander Moisseev <moiseev at mezonplus.ru>
> *Betreff:/Subject:* Re: [Rspamd-Users] csession; 
> rspamd_controller_error_handler: http error occurred
>> On 03.01.2024 18:57, Albrecht Backhaus wrote:
>>> Hi there
>>> 
>>> I am running v 3.7.5 and do get csession http errors frequently - see 
>>> example below
>>> 
>>>> 2024-01-03 16:12:52 #1283389(controller) <80e272>; csession; 
>>>> rspamd_controller_error_handler: http error occurred: Not found 
>>>> 2024-01-03 16:12:52 #1283389(controller) 
>>>> rspamd_http_router_finish_handler: path: /robots.txt not found 
>>>> 2024-01-03 16:12:54 #1283389(controller) <15832f>; csession; 
>>>> rspamd_controller_error_handler: http error occurred: Not found 
>>>> 2024-01-03 16:12:54 #1283389(controller) 
>>>> rspamd_http_router_finish_handler: path: /sitemap.xml not found
>>> 
>>> I checked my config, but I was not able to find the issue.
>>> 
>>> Any help and advice is highly appreciated.
>>> 
>>> Thanks, Albrecht
>> 
>> I guess your controller is exposed to the world, so it is crawled and 
>> indexed by search engine bots.
> 
> Sorry - but I do not understand.  My worker-controller config is:
> 
>> worker {
>>     controller {
>>         count = 1;
>>         bind_socket = "localhost:11334";
>>         enable_password = "REMOVED-FOR-PRIVACY-REASONS";
>>         secure_ip = "127.0.0.1";
>>         static_dir = "/usr/share/rspamd/www";
>>         password = "REMOVED-FOR-PRIVACY-REASONS";
>>     }
>> }
> From my understanding it is not exposed to the public. The log entries 
> in my first post are from the rspamd-log. It looks as if the two files 
> robots.txt and sitemap.xml are missing. I never changed ore deleted 
> anything in that path which is obviously /usr/share/rspamd/www.
> 
> Are the two files expected to be there ?
> 
> I am unfortunately still at a loss ...