Advice on server infrastructure?

Spyridon
edited August 2011 in Photon Server
I've currently got a RPG I've been working on (Unity3d, Stand-alone, not a mobile game) and I've reached a point where I want to start implementing multiplayer functionality, which is obviously going to require some design time on server structure. I've only had little experience with networking in the past so I need some advice. I would like to plan things out, at least loosely, that way I dont make more work for myself in the future by having to restructure everything.

The ideal server structure I would like to have resembles something like how Diablo 2 works - seperate rooms with a fairly small max per room, with the ability to globally chat with any other player, friends list functionality, etc.

Now to my questions...

1) With pricing based on user caps w/ an optional unlimited option, such as 1000 users, followed by unlimited, on each server. But most online game I've played have a cap on users anyway... Even if I get a unlimited license on one server, will I eventually run out of resources and need another server anyway? I know for at least the chat server I'll probably need unlimited once the game grows enough, but for actual game servers would it be better off getting multiple 1000-cap servers rather than a single unlimited? Or with options such as Cloud servers, would you actually be able to support "unlimited" users on a cloud server?

It's hard for me to decide on which pricing plan is appropriate when I'm not even sure if I'm going to need 1 unlimited license or multiple licenses regardless.

2) I know that many online games use separate "servers" for various things, for example a "chat" server, a "game" server, and sometimes an "Monster/NPC AI" server, sometimes an "item" server or "auction house" server, etc. But again, since charges on a per-server basis. Is splitting up servers like this a requirement due to resources? Or (again) does a cloud server option make this unnecessary?

3) I would like to take in to consideration pricing of servers... Depending on the answers to the first 2 questions, I'm wondering what the pricing situation is going to look like. If cloud servers are chosen I hear they can get to be pretty expensive. Is that the most reliable option for this type of game? For cloud servers, do you need to rent them out or is it possible to build your own cloud server? Would building my own even be worth it, considering I would have to invest in not just the hardware but a good enough ISP connection as well?

Aside from that, any other insight you could add based on your answers would be appreciated. I'd like to know what would be ideal for this type of game, not just from an economic perspective (both server and middleware costs), but also from an architectural perspective, as I'm going to be working on the server very shortly and I can't get too serious with my server design without knowing the answers to these questions. I really need to come up with a design plan for my server structure before I can move on.

Thanks.

Comments

  • Just my 2c.

    *Cloud Servers* are just VPS's. They cannot *scale* beyond that of dedicated server (and impose various overheads).

    They have a lot of advantageous, but 'unlimited' scaling doesn't exist unless your code takes into account the numerous VPS's you are running on.

    So unless, I'm mistaken, exit-games see cloud as just a bunch of machines - and requires a license as per normal.

    Not claiming expertise though :)


    You need to know how many players you want to support. The more the harder it is to write the code (e.g. specialized servers as you already mentioned).

    If you support 100 CC users a core comfortably, you could in theory just buy a big 64 core server, and be able to support ~6k CCU. Now if you only expect to see 1k CCU... no need for all that complicated stuff :)
  • NPSF3000 wrote:
    Just my 2c.

    *Cloud Servers* are just VPS's. They cannot *scale* beyond that of dedicated server (and impose various overheads).

    They have a lot of advantageous, but 'unlimited' scaling doesn't exist unless your code takes into account the numerous VPS's you are running on.

    So unless, I'm mistaken, exit-games see cloud as just a bunch of machines - and requires a license as per normal.

    Not claiming expertise though :)

    Thanks for the response. Some confirmation on this would be appreciated. I've seen some conflicting reports about the answer to this question on the Unity forums as well, so if anyone can clear that up :D
    NPSF3000 wrote:
    You need to know how many players you want to support. The more the harder it is to write the code (e.g. specialized servers as you already mentioned).

    If you support 100 CC users a core comfortably, you could in theory just buy a big 64 core server, and be able to support ~6k CCU. Now if you only expect to see 1k CCU... no need for all that complicated stuff :)

    How many players? Well, that depends on how far down the line we're talking...

    Part of the reason I'm going for the room-based server structure is so that the game will work as intended for the 100 CCU of the free Photon, but I'd like to design the infrastructure so that I could expand it as needed. The main thing is I don't want to run in to issues down the line where I find the demand for the game is going up, but my infrastructure was not designed to support any more players without a complete redesign.

    Unlike an MMO server, where you reach a CCU cap and you would have to make a new server that is completely separated from all the other players, I'd like an architecture where I can expand the amount of "concurrent game rooms" that the server is able to support and keep everyone together (similar to Diablo/BNet).

    Not sure the best method of how to do that... Let's say for example a single server is at 1000 CCU, I'm not sure whether it would be easier to expand the total CCU of the server to make the single server support more rooms (probably wont work if you are correct about Clouds), or would it be a better idea to have an architecture where the matchmaking server has a list of Photon servers and design it so I can just throw on one more 1000 CCU Photon server. Or is there any other methods I'm not aware of that would be better suited for this style of game?

    Also not sure if I should just make Monster/NPC AI as part of the "game room" or on their own separate server - that probably depends on which method is best and what the resources look like on each server, but if anyone has advice on what I could expect it would be appreciated.

    I'm also assuming some special attention will be needed for the chat/matchmaking/database servers as they will be "global" unlike the actual game rooms and need to be aware of all players who are online at a time. Will probably need at least one unlimited server to handle that regardless of how I design the "game room" servers.

    One thing I did not take in to consideration is resources on the servers - of course if I find out I can support far more than 1000 CCU easily I may change things accordingly, but I'm looking for the best method to go in to things in order to avoid running in to problems in the future. I would rather take a little more time to plan and design an expandable infrastructure now rather than make much more work later on. I've seen far too many games shoot themselves in the foot and restrict themselves like that.
  • I think you should start with splitting the different servers into different applications and run them with one photon instance. This should make it easier to move the different parts to different machines later-on.
    To avoid that your clients have to deal with x connections you might also want to add a proxy application as dreamora suggested in another recent thread.
    Server resources will limit the ccu eventually, regardless of the license you buy.
    If you target for unlimited scaling you need to be able to scale out every part of your game including matchmaking and chat.
    How many clients one machine can actually serve depends on your game and the hardware.
    If you don't know the max ccu before you can always upgrade later and pay the difference.
  • Thanks for the response.

    So with server resources limiting the CCU eventually, that means the method of having a "master server" that links to a list of "game servers", where each game server can support X amount of clients, would be ideal, as long as I also have a way to scale out matchmaking/chat, correct?
  • I guess you would have load balanced proxy servers that forward certain operations to the master, otherwise even the master server might get too much load at peak times.