Photon server stops randomly for no reason

Options
We are running a photon server instance for our turn-based card game which on average has 100+ CCU playing the game at all times, The server executable is hosted in AWS EC2 Windows instance and we monitor it via AWS Cloud Watch, For many months we were noticing a rare event where the server stops everything for a brief moment, happens once or twice a day, We thought it was something with Cloud Watch and the in-game users are not affected.

But recently our team has noticed say 4 people are in a match and this happens, the photon does not register a disconnection and the server just does not respond and nothing happens until the user exits the match and starts the new match, then everything works as it should. now at that moment of server blackout players are stuck wherever they happen to be matchmaking/in-game/in the lobby, everyone, all 100 gets stuck there,

51186957657_9a07caa08c_o.png

I do have a photon disconnection event system everywhere but even that's not working, I'm clueless about what to look for, server CPU/ram/network uses seem fine, Server load is less than 10% on average on CPU, RAM usage at 30%. Any guess how to resolve this?

Comments

  • chvetsov
    Options
    hi, @isfakur
    please provide as logs. it is not possible to say anything.

    Also you could consider upgrade to new version of SDK. it will be definitely time consuming but it worth it, because one of the issues could be that some one just sends you some special data and your server starts to hang.
    We closed a lot of such issues. also we introduced some ways to control what and how often clients send. So you really might be interesting in this

    best,
    ilya