Garbage Collection

kamend

Hi,
I have a weird problem, we are making a 2D multiplayer platformer, where we have a lot of RPC calls: jumping, using weapons. I noticed that every RPC call allocates around 1kb+ of memory and I am experiencing some weird glitches from time to time, which I think is related to Garbage Collection. I used the Profiled and indeed, we have a large GC spike from time to time. Right now I have made, so GC.Collect is called every couple of frames, so we could take less time, but how do you deal with this situation? Is this normal memory allocation size for RPC calls?

Thanks,
Kamen

kamend

What I also noticed is that PhotonHandler.Update() is allocating 1-2kb+ every 6 frames, which I guess will also cause the GC.Collect to be called often. This is not related to RPC calls, but to how many players do you have in one room.

Tobias

Which version of PUN do you use? We optimized the more recent ones to re-use memory streams, etc. Did you update recently?
When you are up to date and see some memory waste in profiler, let me know where it is and I can take another look of course.

In the end, we can't avoid all garbage: The incoming and outgoing messages have to be read, written and passed from peer to peer. I can try to optimize things of course.

kamend

Thanks for the quick reply, here is a screenshot of a very simple test I made:



These are just two connected clients in one room, one is a OSX build I made and the other one is the Unity Editor. As you see every couple of frames PhotonHandler is taking more time and it is allocating more data (the time is not a problem). I guess this is where the data transfer happens, so my guess it that this is just how Photon works and these allocations are part of the incoming/outgoing data. In out game we are seeing larger allocations around 1-2kb, which I guess is related to syncing 4 clients. At some point the allocations trigger the GC.Collect, which than leads to a slight freeze.

I have uploaded to test project here: http://www.kamend.com/tmp/PhotonTest.zip

I am using the latest Photon version from the Asset Store.

Velo

Hello,

I am having the exact same problem. Huge garbage collection spikes every 10 seconds or so. To the point where my game literally freezes for a second or two. The profiler indicates it is from PhotonHandler.Update(). And then under that: "SendOutgoingCommands".

Whatever is causing the garbage collection is taking up 1096 ms of frame time. It's very noticeable.

Is this something that can somehow be worked around or fixed? Thanks for any help.


Here is a picture from my profiler:

Tobias

@kamend: Sorry for the delayed response. The GDC and some other tasks meant that we didn't look into things yet.
Thanks for the test case. We will run it and check if we can improve things.

We can probably avoid some more memory allocation but there are limitations for the current implementation. We'll see what we can do and we also plan a refactoring of the lower APIs, so we can improve things.
I agree, that the spikes should be sending/receiving every 100ms. How high these are, depends also on what you send.

@Velo: I answered (now) in your other thread, but to consolidate you could reply here (with a link, maybe). The shallow profiling is usually not very useful. You need to deep profile and show as many (relevant) branches open, as possible.

Velo

Hi Tobias,

Thank you for your reply. I did another test with my game and deep profiled it this time. Here are my results and a picture:




It looks like a large portion of it is coming from the Photon Animator View script on all of my units, although the Photon Transform View generates quite a bit of GC as well. The actual main culprit looks to be the resizing of arrays that is taking place, as well as some converting of arrays as well.

Is there any way that either you guys or myself could create "container" arrays (maybe they need to be very large) that could hold the data without needing to be resized? Perhaps this could take place at the very beginning of the game or something. Personally, if it's possible, I'd choose to carve out a bunch of memory before the game begins that photon could use over and over again, without having to trigger the GC. But I really have no idea if that's even possible.

Another thought would be to try to spread out the "workload" over more frames/seconds, so that not all the GC has to happen near the same time. As I have found many times in working on an RTS game, one of the keys is simply not to do everything all at once. But to spread it out over time (if possible).

I found this reference article on Unity's GC helpful: http://docs.unity3d.com/Manual/UnderstandingAutomaticMemoryManagement.html

I'm not ruling out that it could be something that I'm doing wrong as well. I'm personally looking at ways to not have to observe the Photon Animator View at all, since I'm doing an RTS. But even if I did, the Photon Transform View is generating about 1.4k GC allocation by itself, and if say my game has 100 units vs. 100 units (in a 1v1 game), the GC spikes would probably still make my game unplayable.

I know that RTS games are very tough on networking in general, but I was hoping Photon could handle at least a 100 unit vs. 100 unit RTS game. If, in the end, Photon simply can't handle something like that, I'll understand. But I at least wanted to try to solve the issue and give it a good shot.


Thank you for your time, and let me know if you have any ideas, workarounds, or fixes that you can think of.


Thanks!

Tobias

There is room for improvement in your code but also in PUN.

PUN has a lot of features, which are good for "ease of use" but not good for performance. With 200 units, we get into the region where you have to avoid some nice to have features and implement things in a custom, more specific way.

Syncing animations for 200 units is very likely to be "too much". It should not make a big (visual) difference to just send what units do and then trigger animations based on movement and actions. So instead of sending the details of animations, send: "this unit moves from here to there and shoots X".
https://doc.photonengine.com/en/pun/current/tutorials/synchronization-and-state


The PhotonStream is eating most of the memory at the moment. Actually, it should only be in use in the main thread, so it should be safe to re-use it. Then it will grow only once.
If you want to experiment: Find NetworkingPeer.OnSerializeWrite() and in it: pStream. Make that an instance of NetworkingPeer. Initialize it once and reuse it. You need to add a Reset() method to PhotonStream, which clears the stream's internal List (writeData.Clear()). Use it where the new PhotonStream was created.

It still will get copied to an array though. This might be a bit more tricky.

I will have a look and see if I can post a link to an updated version later on.