We are seeing ClientTimeoutDisconnect's in bulk

Hey everyone,

We have started seeing around 300-400% more ClientTimeoutDisconnect's after switching from reliable to unreliable UDP. When we get one ClientTimeoutDisconnect we often get more shortly after until roughly 50% of all players playing have gotten it. We have managed to replicate it and affect how frequently it happens by sending more or less packets and adjusting the SentCountAllowance. Many packets (~20000/s) and SentCountAllowance set to 5 will make it consistent.

In a regular scenario we are sending a packet per player every 100th millisecond no matter whether a packet is dropped or not. The data in a packet will most often have a size of 6 bytes (our data sent in EventData.Parameters), but can contain up to 50 bytes.

The amount of ClientTimeoutDisconnect's are not proportion to how many players we have playing, quite the opposite actually. The players that are getting disconnected are often the players who are the furthest from the server, geographically speaking.

We are using servers from Azure.

Some sparring on this issue would be greatly appreciated.

Comments

  • @kennethharder
    >We have started seeing around 300-400% more ClientTimeoutDisconnect's after switching from reliable
    this is more or less clear. when you use reliable we send till success or timeout. and other packets stuck. so, server has confirmation that client is alive because it is able to get acknowlege from client.
    when you use unrealiable server and client use pings. and for some reason they do not reach server.

    How often do you call peer.Service on client side? What machine did you use on Azure?

    best,
    ilya
  • kennethharder
    edited January 2018
    Thanks for responding so quickly, @chvetsov

    As far as I can deduct it is due to the server not responding to client pings, especially considering it affecting many clients at once (~30 rooms within 10 seconds). While trying to understand when ClientTimeoutDisconnect occurs, we managed to make it happen by stopping packets from being sent from the server.

    We call peer.Service in Unity update.

    We use the Standard D3 v2 (4 vcpus, 14 GB memory) with Windows Server 2012 located in West Europe.
  • are you performing some load tests?

    best,
    ilya
  • Hey @chvetsov, not initially, however, in an attempt to figure out what is happening we are trying to replicate it locally.