Number of Players Supported by Photon Cloud?

Kuroato
edited March 2015 in Photon Server
How many players can Photon Cloud Handle and still Perform pretty well?

I have a project I worked on previously where I limited the Game Sessions to 10 Max Players, and if I reached the Max Players of 10, the Game would Lag like Crazy!

I was never sure if it was the Code, or it was Photon Cloud Server not able to Handle more than a Few Players...

What is a Safe Number of Players per Game Session should we allow when using Photon Cloud?

Comments

  • Kaiserludi
    Kaiserludi admin
    edited April 2019
    For the whole game: There is no limit: We have customers on Photon Cloud that have tens of thousands of players of their game online at the same time.
    For single games: There is no hard limit, but the room logic of the LoadBalancing layer that runs on top of the Photon core on the cloud (and also of the applications Lite or LiteLobby that one could alternatively choose on a self hosted servers (although they are rather deprecated, because they are limited to one single server and do not offer anything that LoadBalancing doesn't also offer.)) doesn't ship with any kind of interest management. That means: If you want to put more than let's say 16 or so players into a single room so that they can interact with each other you will have to implement some kind of interest management in you game logic to make sure that messages are only sent to those players to which they are of interest (for example you won't have to send position updates several times a second to players that can't see your player anyways as they aren't in visual range). All true open world MMOs have some kind of interest management, as no hardware on earth now or in the foreseeable future is able to handle the amount of data that would get sent around if 100.000 players send 10 messages a second to 100.000 other players, each (that would be 100.000 times 10 times 100.000 == 100 billions of messages a second that get sent around).
    The performance bottle neck that will by far be hit first here is the client-side, especially older mobile clients, not the server, so a switch between Photon Cloud and Photon Server won't make a difference.

    The default receiver list of a message consists of each player inside the same room, but the sender of the message, but you can freely specify your own lists of players-ID, player groups or interest groups on the client-side to implement an interest management in you game logic that makes sense for your game (sadly the right solution for interest management is highly game logic specific so we can't really offer an out of the box solution for that), while still not needing custom server logic and therefor still being able to use Photon Cloud.

    However Photon Cloud limits the amount of messages per room to 500, which is more than enough for i.e. 4 players per room, but if you want hundreds of players in a single room then even with the best interest management you would still by far exceed that limit.
    That limit isn't a hard limit that gets enforced technically. We don't just cap your game and throw message limit errors or something like that, but we just monitor how many messages an application sends and if you exceed the limit by too much for too often then we will contact you to find some solution.
    The reason for this limit is: more messages mean more traffic, which is the main hosting cost factor, so with a higher general limit we would have to increase the prices. However if a game has an accordingly suitable interest management, then 1.000 players in one room won't create more traffic than 1000 players in 100 rooms, so that 100 times the message limit per room would be fine price-wise in that case, but you would have to contact us per mail for us to find a price model that suits your case, if your game type really needs to put many players into one room and can't split them up into groups of only a hand full of players per room.
  • wow! OK thank you so much for all of this info! very helpful!! Thank you for taking the time to explain all of this really appreciated.
  • Hey sorry to open thin up but I just red this. I thought message limit was 500 messages/s not 400. And Kuroato was having problems with 10 players in the same room... that gets me thinking that it wouldn't be possible to have a normal 5 vs 5 Moba match for example. I'm just thinking out loud here but in the case of a MOBA you will have team fights with the whole 10 players throwing stuff at each other at the same time, and that will need an accuracy that would get messy with a high lag... so... if I'm serializing position and rotation, and sending Q, W, E, R damage through RPCs, and every player throws an attack at the same time in a team fight.... would there be any limitations??

    I'm sorry if question may sound dumb but I think I'm not understanding clearly what IS the message. I'm guessing the serializer sends one message per second per value sent or something like that, if I stick to the 10 msgs/s that Kaiserludi pointed sent by 10 players are 100 msgs/s and 300 left for the RPCs... if all players drop let's say a Q all at the same time, how much would those RPCs cost?
  • Interesting questions, I have the same concerns actually

    Any reply please?

    Don T.
  • About messages (extensive version)
    We limit the messages (updates) per room and per second for two reasons:
    1) Things break when you send too many updates. There is no fixed cap though. This depends on bandwidth, devices, etc.
    2) Make sure everyone has a fixed slice of our shared servers.

    The limit will be 500 msg/room/second.

    Messages are in general anything that updates someone else or does something on the server.
    In Photon terms: All Operations, Responses and Events are messages.
    In PUN terms: RPCs, Synchronization updates, Instantiates, changing Custom Properties are messages.

    Sending an Event (or RPC) to all other clients counts are one message per player in that room. One sends, the others receive.
    PUN is special, because it tries to aggregate updates by OnPhotonSerializeView. If possible, the updates of several objects are aggregated into one message. Also, the observe mode affects this: "Unreliable On Change" stops sending anything when the GO doesn't move between updates.
    In worst case one object can cause 10 messages per second per player. That's not common however.

    We average the messages/sec over the existing rooms and there is no hard limit that breaks your game when you go over it. Keep an eye on your counters and prepare to improve things if needed.

    About MOBA Updates
    With some clever design, you should be able to reduce the number of messages / second dramatically. Actually, it is not extremely useful to send position, rotation and basically everything for every unit X times per second. Like in an RTS, the bulk of units doesn't get a meaningful update each second but only when the user issues some new commands. Then a unit gets a new target, some place to go to, etc.
    When you keep that in mind, you can minimize the number of updates per unit and interpolate between positions, rotate to the new target position, etc. Just like you do locally (no player is rotating his units manually! you do that with "AI").
    The trick is to make those interpolations be correct (on any client) or at least look correct. It's a game! Fake stuff!

    The standard inspiration for anyone who can't send enough messages for X units on a map is this article (it deals with modems but you get the idea):
    http://www.gamasutra.com/view/feature/3094/


    Summary
    While this is a very vague answer in some parts, don't worry too much. Experiment! You will find ways.
    If you're a bit over the limit, we don't care if your game rocks! The limit is there to make sure Photon Cloud can continue to work. It has reasonable limits and others do well with those, too.
  • Amazing Toblas :-), thanks for clear answer.

    I think It could be worth it to dedicate a documentation page in your Reference section to explain what you just explained (basically a copy&paste at least), I think those 500 msg are not clear for many people that start with Photon and might be good to put it there.

    Cheers,
    Don T.
  • Hello,

    I am newbie. I intend to use the PUN to allocate approximately one thousand and five hundred players on a single map. Will be at most fifty sharing the same area of interest. What do you recommend?

    Thank you,
    Francisco Neto
  • How can I measure how much bandwidth my game is using to determine how many concurrent players can I have in a room?
  • Thanks for the clear informations! I missed this.. Now I understand the messages limit.
    But on question for clear the properties:
    You said, that the custom player/room properties are +1 message/s when changing.
    So this mean, that custom property reading is not a message?

    So for example:
    If I have 10 user in a room, I can save 9 message, if I not use event broadcast for all, just set a room property and other 9 user read this property?

    So room/player property
    set = 1 message,
    read = 0 message,?
    It sound very good
  • @kikinhobiaso: PUN is probably not able to cover that. 500 in a room is quite a bit, even if you do interest management. In Photon, the only solution to support this, is to customize the server and implement fitting interest management server-side.

    @Aidin: Keep in mind that there is probably not a 1:1 relationship between bandwidth and players. You can measure it with the usual tools: Wireshark, etc.
    Also, read the section about Traffic Stats:
    https://doc.photonengine.com/en/pun/current/reference/analyzing-disconnects

    @Balazs: Once a property update got broadcast, it's not a message per read anymore (as it's cached). But setting a property, causes one "send" message and "update events"*player. You can set multiple properties in one go and it still is one message per player (one sends but does not receive).
    When players join, they get the current properties in one message (aggregated).
  • Tobias said:

    @Balazs: Once a property update got broadcast, it's not a message per read anymore (as it's cached). But setting a property causes one "send" message and "update events"*player. You can set multiple properties in one go and it still is one message per player (one sends but does not receive).
    When players join, they get the current properties in one message (aggregated).

    Hi, Tobias. I'm using PUN. I'm wondering could the "update events (receive)" be separated from the "send" message.
    For Example, There are A, B, C three player in the room. Default situation: 0.1s A changed a property, 1 send + 2 updates = 3msg. 0.4s B changed a property same 3msg 0.9s C changed a property 3msg. 1s time's up, Total 3+3+3= 9msg last second. If I want the game only synced per second. Ideal situation: 0.1s A changed a property, 1 send + 0 updates, 0.4s B changed a property 1 send +0 updates, 0.9s C changed a property 1 send + 0 updates. 1s time's up All player(A, B, and C) updates All properties, 3 msgs. Total 1+1+1+3 = 6msg last second.
    I think more the players this way can save more "update events (receives)" I'm wondering how to do that manually in PUN? What method to call that only send "PropertyChange" and what method to call to update the properties. And the "get the current Properties in one message" means all Player's Custom Properties and Room Custom Properties in one message?
  • Hi,

    Is there anyone can advice me, is photon capable to handle 100 players on single level-map?
    With which photon varient should we go for? (PUN/Realtime/Clound/Bolt)

    We're planning to make action FPS/TPS game for mobile platforms.
  • We are facing a similar problem, we are currently using PUN with Photon Cloud.
    I don't think that Bolt will work as it requires one client to be the server and if all of your clients are mobile phones things may not go well.
    Pun vs Realtime shouldn't make much of a difference from what I understand, PUN is just a unity wrapper for Realtime.
    Cloud vs On-Premise Server is an interesting question. If you customize your server you may be able to optimize bandwidth or reduce client overhead.
    However we have not yet tried hosting our own servers and we are not sure which cloud VM provider would be ideal, and what it would cost to match or exceed what photon Cloud provides.
    I think that Photon Cloud uses Azure. (try pinging ns.exitgames.com)
  • @bluebird, Photon is just a transport. We are able to handle up to 5K users on the cloud on one server. but depending on what and how many packets you are sending this numbers may vary. if you have one level with 100 players and control traffic basing on distance then everything should work good

    @DavidSWu you are right. ns.exitgames.com is on Azure. but other servers are not. We use SoftLayer, GCore and other providers to get a better experience for our customers

    best,
    ilya
  • Kaiserludi
    Kaiserludi admin
    edited April 2019
    @DavidSWu:
    No out of the box solution is really suitable for such amount of players inside a single room as without any interest management the amount of messages increases squared to the amount of players and it isn't really possible to implement a good general purpose interest management as what works well depends highly on the specifics of each game. Hence this is something that the game developer himself has to solve.
    You can implement quite a bit of client side interest management for which you don't have to add or modify server-side code.
    Here is a blog post by the developers of a Photon powered games about how to do this: https://www.m2h.nl/network-traffic-culling/

    If this kind of approach on its own is not enough for your use case and you require something more sophisticated, then you might at some point run into the need of running custom server side code.
    Custom server side is not supported by Photon Public Cloud, but it is supported by Photon Enterprise Cloud and by self-hosted Photon server instances.

    If you should decide to go with the self-hosted route, then my answer to the question "which cloud VM provider would be ideal" would be: "None of them." When it comes to traffic-intensive usage like game servers, Virtual machines have a considerably worse performance to cost ratio than bare metal machines and you would end up with paying a lot more on hosting costs for the same amount of players with Cloud VMs than with bare metal machines. Hence it makes more sense to use bare metal machines to handle your normal amounts of CCU and to only use VMs to handle peak CCU. The main advantage of Cloud VMs is that you can get a lot of additional servers on really short notice and only rent them for a couple of hours, while bare metal machines usually require you to rent them for at least an entire month and might need several days to become available. For this reason Cloud VMs, although a lot more expensive per CCU than bare metal machines for capacities that you need the whole month day and night, can be a cheaper way to handle to handle additional capacity requirements that are only present for just a few hours or days or to quickly react to an unexpected general increase of capacity requirements, as with bare metal machines you would have to keep most of the time unused reserve capacities around just in case.

    As @chvetsov has already mentioned, Photon Cloud also uses bare metal machines under the hood. We would need to charge our customers considerably more if we would use Cloud VMs for all of our servers.

    When comparing prices of different hosting providers, focus on the traffic cost. For game servers, how much traffic is included and what you are paying for additional traffic is really the main cost factor (aside maybe from the salaries of your server operations employees) and when you look at just the price of the rented hardware itself without considering traffic, then you will most likely end up with a nasty surprise in form of a bill that is several times a big as you might have expected.