[Fault Tolerance]Any suggestion when game server crashes

Options
garric
garric
edited March 2012 in Photon Server
Hi,
Haven't found any doc on Fault Tolerance using Photon, is there any suggestion?
Imagine, if one of our game servers was crashed for some reason, we had to instantly recover it.
So using hot backup system? I mean, when crashes, just redirect clients to a mirror of this game server?
Or update properties of this game to a database? I mean, When crashes, just restore game server with data on database?

Or there has some existing middleware can solve this problem?

:)

Comments

  • Philip
    Philip ✭
    Options
    Hi garric,

    fault tolerance is a complex issue and depends a lot on your game, your architecture and how much you are willing to invest ... anyway I can tell you about the strategy we follow on our cloud service a highly available service. The system is based on a set of loosely coupled systems:
    a. the master or lobby service
    - game server report games and health/load-level
    - player connect to create/join games
    - the master "sends" players to game server according their load
    b. the authentication service
    - used by master to authenticate applications
    c. the set of game servers
    - clients connect to "play"

    Each of them can fail without disrupting the others and they are "self healing" when they recover.
    If the master crashes - a hardware loadbalancer fails-over to a standby master.
    The game servers "reconnect" to the master and send their list of active games.
    If a game server fails - the master takes it out of the list of games - and stops sending
    clients to it.
    If the authentication service fails - which is only called on application i'ds not found in the cache
    we assume the authentication is OK. This is a risk we decided we could take because we
    made sure the service a failed-over webservice and a failed-over database won't fail.
  • garric
    garric
    Options
    @Philip
    err, it seems more confused to me now :?
    "The master server 'send' clients to game server with min-workload", so the game server is hosted by ourself, or hosted also in cloud?

    If the game server is hosted in the cloud, what about the specific game logic, say a playcard turn-based game? In the load balancing project, the GameClientPeer wil treate all the custom operation as unknow operation and directly response to client with the message "Unkown operation code...". So this sounds that, the game server will be hosted by ourself, is that right?
    :)
  • Philip
    Philip ✭
    Options
    garric wrote:
    @Philip
    err, it seems more confused to me now :?
    "The master server 'send' clients to game server with min-workload", so the game server is hosted by ourself, or hosted also in cloud?

    I described the architecture of the cloud: http://cloud.exitgames.com.
    garric wrote:
    @Philip
    If the game server is hosted in the cloud, what about the specific game logic, say a playcard turn-based game?

    I was talking about the game server that are part of the cloud and don't support custom logic.
    garric wrote:
    @Philip
    In the load balancing project, the GameClientPeer wil treate all the custom operation as unknow operation and directly response to client with the message "Unkown operation code...".

    Correct
    garric wrote:
    @Philip
    So this sounds that, the game server will be hosted by ourself, is that right?

    No. We currently don't support a mixed model (master on the cloud and game server selfhosted).
    If you need custom game logic you have to host your own master too.

    In this thread I explain the model we recommend to integrate custom logic like persisting profiles
    viewtopic.php?f=5&t=1514

    The case of a turnbased playcard game you mention ... well the cloud is targeted rather for a design where you have this logic in one of the clients a "master-client" (for instance the creator of the room). Of course you then have to deal with "master-clients" leaving/disconnecting ...
  • Philip
    Philip ✭
    Options
    @garric: please let us keep the discussion on supported models/workflows of the cloud on your other thread viewtopic.php?f=5&t=1514

Cookies,
anyone?

We use cookies and related technologies to enhance your experience, show you personalised content and analyze performance and traffic on our website. By clicking on the „Accept All“ button you consent to the use of non-functional cookies and the subsequent processing of personal data to optimize our website and services as described in more detail in our Privacy and Cookie Policy.

By clicking on the „Customize or Deny all“ button you can decide otherwise. Clicking on the „Customize“ button will take you to a page where you can configure the usage of non-functional cookies (and related technologies) or deny all of them. You can access these settings at any time and also subsequently deselect cookies at any time in the footer area of our website.

ACCEPT ALL

Cookie Overview

We use the following categories of cookies and related technologies to enhance your experience, show you personalized content, and analyze performance and traffic on our website. We respect your right to privacy and accordingly you can chose to not allow some types of cookies (and related technologies). Click on the different category sliders and change our default settings to manage your cookie settings.

For more information on the specific cookies/related technologies we use and on how we use these, please see section 15 E. of our Privacy and Cookie Policy.

ABSOLUTELY NECESSARY

Authentication cookies we use are required to run our services … Cookies are required: Link

FUNCTIONAL AND MARKETING COOKIES

These cookies collect anonymous data and allow us to optimize our website and user experience. These cookies are listed here:

ANALYTICS

Help us to understand how visitors interact with our services, enables us to analyze and improve our services (also through third party analytics).