In Progress General 2 failed health checks kill a server

[takeoshigeru]

Hi, I'm hosting a server in China so the connection to the TaleWorlds server is very unreliable and after two failed health check my server gets disconnected even though it's perfectly healthy.

[22:40:29.967] Exception #0: An error occurred while sending the request. The response ended prematurely. ||| StackTrace: System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: The response ended prematurely.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at System.Net.HttpWebRequest.SendRequest()
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
[22:40:29.968] Exception #1: An error occurred while sending the request. ||| StackTrace: System.IO.IOException: The response ended prematurely.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
[22:40:29.968] Exception #2: The response ended prematurely. ||| StackTrace:
[22:40:29.968] Http post request(TaleWorlds.Diamond.Rest.AliveMessage) will try again, iteration count: 1
[22:40:30.012] FPS dropped below 60: 59.499020
[22:40:30.971] Retrying http post request, iteration count: 1
[22:40:31.078] Http Post Request to https://bannerlord-even-custombattleservermanager.bannerlord-services-2.net:443/Data/ProcessMessage
[22:40:36.084] Http post request(TaleWorlds.Diamond.Rest.AliveMessage) is successful with iteration count: 1
[22:40:36.084] Setting current request message as failed because server returned unsuccessful response(ResultFromServerTask)
[22:40:36.085] ClientRestSessionTask::SetFinishedAsFailed::TaleWorlds.Diamond.Rest.AliveMessage
[22:40:36.085] ClientRestSessionTask::SetFinishedAsFailed:: TaleWorlds.Diamond.Rest.AliveMessage done
[22:40:36.093] Shutting down in 20 seconds with reason 'Disconnected from custom battle server manager'

Anything I can do about it?

 

[takeoshigeru]

What's weird is that it seems like it's supposed to retry 5 times by reading ClientRestSessionTask code but there is no 5 retries in the logs

[20:21:18.217] Http Post Request(code:NoCode) with message(TaleWorlds.Diamond.Rest.AliveMessage) failed. Retrying. (UnknownError)
[20:21:18.230] Exception #0: An error occurred while sending the request. The response ended prematurely. ||| StackTrace: System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.IO.IOException: The response ended prematurely.
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at System.Net.HttpWebRequest.SendRequest()
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
[20:21:18.231] Exception #1: An error occurred while sending the request. ||| StackTrace: System.IO.IOException: The response ended prematurely.
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
[20:21:18.231] Exception #2: The response ended prematurely. ||| StackTrace: 
[20:21:18.231] Http post request(TaleWorlds.Diamond.Rest.AliveMessage)  will try again, iteration count: 1
[20:21:19.323] Retrying http post request, iteration count: 1
[20:21:19.431] Http Post Request to https://bannerlord-odd-custombattleservermanager.bannerlord-services-2.net:443/Data/ProcessMessage
[20:21:19.643] FPS dropped below 60: 47.875751
[20:21:20.200] Http post request(TaleWorlds.Diamond.Rest.AliveMessage) is successful with iteration count: 1
[20:21:20.200] Setting current request message as failed because server returned unsuccessful response(ResultFromServerTask)
[20:21:20.200] ClientRestSessionTask::SetFinishedAsFailed::TaleWorlds.Diamond.Rest.AliveMessage
[20:21:20.200] ClientRestSessionTask::SetFinishedAsFailed:: TaleWorlds.Diamond.Rest.AliveMessage done
[20:21:20.201] ClientRestSessionTask::SetFinishedAsFailed::ResponseCustomGameClientConnectionMessage
[20:21:20.201] ClientRestSessionTask::SetFinishedAsFailed:: ResponseCustomGameClientConnectionMessage done
[20:21:20.201] ClientRestSessionTask::SetFinishedAsFailed::PlayerDisconnectedMessage
[20:21:20.201] ClientRestSessionTask::SetFinishedAsFailed:: PlayerDisconnectedMessage done
[20:21:20.207] Shutting down in 20 seconds with reason 'Disconnected from custom battle server manager'

 

[MArdA TaleWorlds]

Forwarded to the QA team for further investigation. We will reach out again if we need more information. Thanks for reporting and sorry for any inconvenience!

 

[takeoshigeru]

Just to explain the criticality of this issue, this is the number of players on the cRPG chinese server. You can see every ~30 minutes there is a drop because the server gets disconnected.