Can I go over 500 msg/s if I lower the data size?

I found a couple of forum discussions / answers that the 500 msg/s is not a hard limit. However I haven't quite understood why that is — and more importantly, when I can go over it.

To give some context, I am working on a team-based top-down shooter and was hoping to make it a 5-vs-5 game. But I completely missed the msg limit (this being my first PUN game). The game was coded to be very low data. It's only sending 2x Vector2 sterilised variables per player (4x floats), and projectiles are also only in 4 floats + a double timestamp (RPC). The game also holds great at a sterilised step rate of 8 and 16 for RPCs, and since most weapons only fire at a rate of under 4-6 bullets per second I should be fine... but I am pushing it very close to the limit.

I mean, just sending the sterilised data will mean I'm right next to the limit at 8 players, and way over at 10 players:
8 (players sending msg's to) *7 (other players at) *8 (step rate) = 448 msg /s
10 (players) *9 (other players) *8 (step rate) = 720 msg /s

And at worst if everyone is calling all 16 RPCs per second:
8 (players) *7 (other players) *16 (step rate) = 896 msg /s
10 (players) *9 (other players) *8 (step rate) = 1,440 msg /s

To summarise my question, how big of a problem will this be for me going forward?
If I keep all data under 10 floats per msg can I get away with 1000 msg/s?
What if I make the game less precise and do 10-12 shorts (or my own custom byte array, of under 64 bytes per msg), can I get away with 1000 msg/s?

Comments

  • After 5 days of digging around I still haven't found an clear answer on this topic — or a solution for lowering my msg/s (though it seems I am rarely going over, mostly staying around 450-480 on average).

    The only thing I found remotely close on the topic of going over 500 msg (but vaguely answers) was a discussion about how some games clearly go over the limit and how these situations are handled on a client-by-client basis by the Photon Team. The vaguely answer being that you should take into account the possibility of having to pay for double the CCU — which sounds fair, but would be great to get a confirmation from someone.

    Really appreciate if anyone can shed some light on this. Getting close to a private Alpha launch and I am not sure if I should start looking for an alternative networking solutions...
  • Hey there,
    not a pro on the issue but i do not think there is a way to have more msg/s by lowering datasize.
    I guess the reason for this is the overhead created by each message itself. Even if the content is empty.

    If you want to have more send rate in your programm you should look into network culling. Basically you try to evaluate if some updates are not neccessary for certain clients and skip the updates for them. (For example a player that does not look at you you perhaps only need half the position updates or try to skip updates when no change to the previous message occured)
    In average you can free enough messages like this to increase send rates at other points.
    Keep in mind that you still have to be able to handle situations where for example all players might end up in one spot without hitting over 500 msgs.

    Depending on your game you might also go ahead and introduce a "server" to your system to get away from the peer to peer system.
    This way you have your masterclient as "non player participant" and only send updates there. Then you can scamble the data there as needed and distribute it to each player again. So each update from "server" to client is an collection of all the updates of all other players combined in few messages.
    Be aware though that latency times will increase like this. Also this solution requires some increased data compression to not hit the maximum message size.
  • I get it. I have considered doing it more like a server approach. Do 7 msg to the master client from each player then the master client send 8 msg, one for each player (15x multiplier of whatever your send rate is), instead of doing 8 players sending 7 msg to everyone else (56x multiplier). Unfortunately the delay will be too great for our game mechanics.

    For culling the problem is the game is an arena shooter, so there is very little chance of things happening out of frame that will not affect the game. I will look into it, but I hope I am not wasting time... since I'm sure there will be moments when everyone will be on screen for more then a couple of seconds. Thanks for the advice.

    So does this mean that you can't go over the limit even if you are willing to pay for the extra usage?

    I love PUN (especially after using it for months)... and it work so well... I would hate to have to switch to different service. The quickest fix would be to just limit the rooms to 6 players, but that will literally kill the entire concept of our game :(
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited December 2018
    Hi @liviu,

    Thank you for choosing Photon!

    though it seems I am rarely going over, mostly staying around 450-480 on average
    You should be fine and you don't have anything to worry about.
    the possibility of having to pay for double the CCU
    No. Where did you get this information from? What you could be paying is the overages of the traffic, by default you get 3GB/month/CCU if you exceed that you pay the difference. See pricing page.
    So does this mean that you can't go over the limit even if you are willing to pay for the extra usage?
    No.

    Hi @Captain_Pineapple,

    to get away from the peer to peer system.
    I just wanted to clarify, all Photon products have a client-server architecture. However, Photon Server by default is a relay server.
  • Hi @JohnTube , thanks for your reply.

    though it seems I am rarely going over, mostly staying around 450-480 on average
    You should be fine and you don't have anything to worry about.

    Last night I did do a more rigorous test and at 10 players I do go way over, sitting on an average of 600-700 and peaking at 800. Of course, not all rooms will have 10 players, and we are considering slowing the re-spawn rate so that we won't be with 10 players alive and moving all the time...

    Will this be a good compromise: if we only go over in some of the rooms and only for a brief period after the match starts?

    @Captain_Pineapple did get me thinking about about how I could do network culling in a way I did not consider: limiting the send-rate as opposed to not sending to everyone (duh! feel so stupid now). I'll look into dropping the send-rate to something like 4-5 when it's out of frame, so you still get everything just not smooth movement. This might work well for us, or at least keep it 90% of the time at under 500 with 10 players in the room.

    For example such a solution could be that the developer subscribes to twice the amount of CCU that he actually generates and in exchange is allowed to generate twice as many msg/s per CCU than he would otherwise be allowed to.


    Above is a quote regarding paying for double the CCU. Link to the discussion below https://forum.photonengine.com/discussion/comment/41783#Comment_41783

    I just wanted to clarify, all Photon products have a client-server architecture. However, Photon Server by default is a relay server.

    On this note, I'm still a bit confused about some of the differences between each service, maybe you can help me get a better understanding.

    I'm assuming the 500 msg limit is due to the shared-server across multiple clients (which ensures the low prices you guys offer for PUN). Is Photon Server service an option for our game? -- is Photon Server the equivalent to what VPS is for web-hosting? (so we pay extra to be moved from the shared server to a private server, pay $95/mo for 500 CCU for licence + whatever the server costs will be)?

  • JohnTube
    JohnTube ✭✭✭✭✭
    edited December 2018
    Hi @liviu,


    Last night I did do a more rigorous test and at 10 players I do go way over, sitting on an average of 600-700 and peaking at 800. Of course, not all rooms will have 10 players, and we are considering slowing the re-spawn rate so that we won't be with 10 players alive and moving all the time...

    Will this be a good compromise: if we only go over in some of the rooms and only for a brief period after the match starts?
    I see these are hypotheses. I think you need to make some tests with a good amount of users/games to know what you are dealing with first.

    Above is a quote regarding paying for double the CCU. Link to the discussion below https://forum.photonengine.com/discussion/comment/41783#Comment_41783
    I see. I think what my colleague @Kaiserludi meant is that one option is to upgrade your Photon Cloud plan for your application to get double the CCU (e.g. from n CCU/month to 2*n CCU/month) and not paying double the price per CCU (e.g. from X$/month for n CCU/month to 2*X$). @Kaiserludi can always clarify that.

    Is Photon Server service an option for our game? -- is Photon Server the equivalent to what VPS is for web-hosting?
    Usually, what we refer to by "Photon Server" or is the self-hosting option. But technically Photon Server is what we run on Photon Cloud and manage for you. It is always an option, yes. But you should know that there are slight differences between what you have on Photon Cloud and what Photon Server SDK offers (some Photon Cloud only features: e.g. Virtual Applications + some configuration differences). Also, Photon Server SDK is lacking behind so some fixes and features are not on Photon Server SDK yet but will be in the future updates.

    With Photon Server also you should do all operations yourself and handle hosting, configuration, traffic, load...the tricky part is estimating what your servers could handle in the number of CCU / Rooms / Traffic etc.
  • Hey, let me repeat my post from the other linked thread here.

    - we do not have a hard cap on 500 msg/s per room, so you can go over a bit
    - when going over the limit
    (i) pay extra close attention to your used traffic (and check traffic overage cost)
    (ii) make sure your clients can cope with the amount of messages (especially mobile clients might get issues here)

    As long, as your higher msg/s rate does not have a negative effect on other apps or consumes too many resources (server CPU, mem., I/O), we are ok with it. If we think your app has a negative effect on others because of increased msg/s, we will ask you to adjust it.

    That said, @liviu with the numbers you gave, you should be ok. BUT, be sure to check your traffic consumption.
  • @JohnTube @Markus thank you both for the quick reply.

    I see these are hypotheses. I think you need to make some tests with a good amount of users/games to know what you are dealing with first.

    We are ramping up towards launching a public Alpha specifically to test the game in a real world scenario. So I should know more in the next few weeks.

    Photon Server is what we run on Photon Cloud and manage for you. It is always an option, yes.
    With Photon Server also you should do all operations yourself and handle hosting, configuration, traffic, load...the tricky part is estimating what your servers could handle in the number of CCU / Rooms / Traffic etc.

    Got it. I'm sure all of these are discussions to have once you get there and probably not something to worry until you have tens of thousands of users, but just wanted to get a better understanding of what to expect and what options we have. I guess the importance part (for us) is having that security that in case of high user traffic (or above then average msgs/sec) there is some easy(-ish) way to fix by moving to a dedicated server / custom plan.

    That said, @liviu with the numbers you gave, you should be ok. BUT, be sure to check your traffic consumption.

    Cool. Yes of course, based on our estimates we should be good on traffic. We will find out more in the following weeks.

    Thanks again to you both!