Socket timeout error on NSCP-0.6.2.1-x64.msi

[jegasmile]

Hi Team,

We have used NSCP-0.6.2.1-x64.msi on our infrastructure. We experienced server often getting socket time our error from various agents with below error. Can you please help us to fix this issue.

2025-03-18 17:43:50: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-03-18 23:48:34: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-03-19 05:44:31: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-03-19 11:44:45: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-03-19 11:46:21: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)�

Server side we get below error

CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.

Thanks,
Jegadeesh

[mickem]

Does it happen all the time or just some times?

Just to make sure:
There was an upgrade issue (though I think that ONLY affects version 0.6.4 to 0.6.9) that cause the config to get replaced with default config so ensure your config is correct.

But a more likely issue is a timeout miss configuration.
What I think happens here is the server has a timeout of 10 seconds, and the client has a timeout of >10 seconds.
And the commands takes more than 10 seconds to execute.
Then after 10 seconds the server closes the connection cause a write failure when the reply is sent.

So what commands are you running?

You can read more about timeout configuration here: https://nsclient.org/docs/faq/#timeout-issues

[jegasmile]

This is not happening sometimes and network team confirmed there is not network connectivity issue
We have same configuration across all the servers only few servers have this problem.

This is the command we are executing
$USER1$/check_nrpe -2 -H $HOSTADDRESS$ -c check_win_eventlog -a $ARG1$ $ARG2$

We are not setting any timeout in both sides.

Thanks,
Jegadeesh

[mickem]

The default timeout is 10 seconds ( https://support.nagios.com/kb/article/nrpe-check_nrpe-socket-timeout-after-n-seconds-617.html ) which is what your seeing so in this case I would try increasing that a bit. eventlog can take time if it is a big log.
Another option is to create a bit more narrow filters using for instance scan range: https://nsclient.org/docs/reference/windows/CheckEventLog/#time-and-date

[jegasmile]

Thanks for the command. we will apply the changes in to our env and update you.

[jegasmile]

We have added below timeout in our client and server.

Client side:

[/settings/NRPE/server]
timeout = 30
check_nrpe -t 30

Server side:

command_line $USER1$/check_nrpe -2 -H $HOSTADDRESS$ -t 30 -c check_win_eventlog -a $ARG1$ $ARG2$

Lets monitor it for a while and update the sttus here.

Thanks,
Jegadeesh

[mickem]

Great, let me know how it goes...

[jegasmile]

Hi,

We are still receiving this alerts from the place where we enable 30 sec timeout as well as other agents as well. We got below exception in Agent side. Can you please advise us.

2025-04-02 05:45:13: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-04-02 11:42:51: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-04-02 11:44:24: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-04-02 17:44:53: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-04-03 05:43:33: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)
2025-04-03 05:44:12: error:D:\a\nscp\nscp\include\socket/connection.hpp:139: Socket was unexpectedly closed trying to send data (possibly check your timeout settings)

Thanks,
Jegadeesh

[mickem]

Well, timeouts are not a "magic solution" it is one tool of many.

What I can see from the timstamps is that the problem occurs at 05:45, 11:45 and 17:45 that is oddly specific.
Do you schedule the check infrequently? Or is it only at these times there are problems (and at other time the check on this specific machine works fine?)
So given this my guess would be that you are running backups or other heavy tasks which takes a lot of resources making the machine respond slower during these timeframes.

But in general if something takes long: it takes long. The question is how long is long and is it acceptable?
How long does it take to run the check when you run into these errors on the machine in question if you run it manually?
An eventlog check can take minutes on a heavy loaded machine with hundreds of thousands of event log messages.
If the check when run manually takes 45 second set the timeout to 60 seconds, if it takes 5 minutes, you likely dont do not want to set the timeout to 6 minutes so instead evaluate other options.

So if your checking the event log maybe you can do that during "off time" i.e. when your not running heavy load?
Let say the job is started at 11:45 and runs for 15 minutes, schedule the check to run at 12:15 instead.

Another option is to check a narrower timeframe in the eventlog (default is 24 hours) but if you check frequently you only need to check since the last or possibly next to last check so if you run this check every hour you could likely set scan-range=-2h and reduce the time it takes to execute?

But all this is tradeoffs that has to be tailored to the specific machine and use pattern so there is no one-size fits-all solution here unfortunately.

[mickem]

Closing this, please reopen if you still have problems...