True Sync for Real-Time multiplayer mobile game?

Hello fellow developers,

I'm currently develipong a multiplayer game using Unity3D (you can check my develpment blog here https://twitter.com/SteelWarsRoyale ) and I have some doubts abot how to implement the multiplayer.

I'm trying to decide a network architecture based on three things:

As for now, I have a first working version implemented using the architecture I-don't-give-a-flying-fu**-about-sync, which means I send player actions (commands) over the net with Photon RiseEvents, the other client receives it and executes the logic of that command. It "works fine" but it's not a robust system by any means.

Small sample video:

The problem is, If I spawn a Unit at time X and there is 150ms of latency, the other player will receive the command and spawn the unit 0.15 seconds later. A small difference between spawnTimes can cause a Unit to act different and create a snowball effect where each client sees a different version. I lost the sync between clients a couple of times doing some simple tests.

The thing gets even worse if you add Physics to the equation. Unity3D Physics are non-deterministic so again, the desync is almost ensured, even if the physics ammount is small (units pushing each other a bit when they accumulate in one lane). At the moment I don't have physics but I plan to add them, otherwise units stack on each other.

I have been reading about Unity TrueSync, which uses a lock-step architecture (pretty common in RTS were you only replicate player actions, like my game) and it also has deterministic physics, which would be great for my game.

The only problem is, I don't trust using lock-step architecture for a mobile game with unpredictable connections, mostly because it "locks" the interaction until messages are received, making the game unresponsive at some times.

Has anyone experience using True Sync in real time games for mobile or any hint about how to sync both clients without going full authoritative external server?

Comments

  • I think I have a solution in mind for for Units Spawn delay problem.

    Clash Royale has a timer for enabling units after spawn, which means that you send the Spawn command and the Unit appears in the map, but it is not enabled until the Spawn Time finishes (between 2 to 4 seconds usually), only then the Unit starts to move/attack.

    I though I didn't need that game mechanic in my game, easy to implement but I didn't feel like doing it. So in my game when you click the Unit button, it spawns innmediately. That is where the latency problem comes. I have no margin on the other client to fix/correct the delay issue. So maybe clash royale did it not only for game design purposes but also because of networking.

    If I add a Spawn delay (it has to be bigger than the average latency to be able to compensate), I have time to compensate the delay. If I have the Time synchroniced between clients, and each client sends the command with a time-stamp, I can calculate the delay of the command.

    If the other client recieves the Spawn command with lets say 150ms of delay, I just have to enable that unit 150ms faster to compesate it, and, if I'm not wrong, the Unit will enable at the same time in both clients.

    Same happens with spells (say throw a fire ball), it can't be instant. Since you click the spell until it reaches the objective, it needs to have a fixed trayectory time. I could compensate it too making it reach the objective X ms sooner, where X is the delay of the packet with the command.

    The bigger I make that Spawn delay time, the more reliable it will be. If I make a minimum time of 2 seconds, that means I can tolerate lag skypes of up to 1.99 seconds. Am I right or I'm building castles in the sky?


    It doesn't fix the non-determitistic Unity Physics once I add them tho :sweat:
  • Hi @MaRTiDoRe, you got the feeling about build a multiplayer game like that. You faced very common problems but there are other like float calculations that drive you crazy, even if the delay time was big enough all the time you could face some tiny variations of trajectory or multiplications and get that snowball effect. This is a very great source to read about network on games https://gafferongames.com/post/introduction_to_networked_physics/.

    TrueSync has some mechanisms to avoid the wait for other players inputs, as some delay window and a rollback system where you can create fake inputs for the remote player and fix the simulation when the real inputs arrives (giving more time to wait for inputs). The main drawback we have today is physics simulation performance, if you have several units it will be impact on the FPS. We are working and modeling our new release to better handle those performance issues, for example next release we will drop support for 3D physics, at least in the first versions.
  • MaRTiDoRe
    MaRTiDoRe
    edited December 2017
    Hello @JeffersonHenrique,

    Wow, what an excellent article! I'm sure you guys re-read that article at least 10 times while developing TrueSync.

    You might also want to take a look at this article, it's pretty good: http://www.tundragames.com/minimizing-the-pain-of-lockstep-multiplayer/

    I implemeted the delay compensation in the unit spawns successfully, now it doesn't matter the latency, units espawn at the same time in both clients. But there still seems to be fractions of miliseconds of difference, and those fractions can make the difference in my game.

    I recorded an example. In this video you can see how units spawn, explosion etc. happens in the same frame (I paused the video several times to check), but even then, there is a des-sync between the clients. The average delay for clients ranges from 60 to 90 ms.





    In the right client 2 melee units engage in combat, in the left client those same units don't engage because they were not in rage. Btw I'm using collider spheres in Trigger mode as "attack range" with OnTriggerEnter.

    So, even compensating commands delay with server timestamps and without physics, I can't have a synced game because there is still microseconds difference, or that's what I think.

    From here I can do two things:

    1- Make the Slave Client (the one joining the room) a render dummy with 0 authority and make the master client fully auth. Units spawned in the slave client won't have Authority to target or die.

    When a unit engages in combat with anoter unit in the Master Client it will send an event to the Slave Client to force that attack.

    Same happens with Units deaths, units reaching the barrier etc.

    The problem: Master Client (the one that creates the room) will have an awesome experience with 0 delays, 0 jittering 0 strange thigs. But the Slave Client, because of the delay, can have WTF situations like:

    "WTF? Those melee units are touching each other but not attacking?" because the attack event for those units as not arrived yet.
    "WTF? that unit reached the enemy barrier but the eney health didn't change?"
    "WTF? That unit's health is completely gone and is till getting hits without dying? Becase the Die event again is delayed.

    May be if the latencies don't go above 200ms it is barely noticeable and credible for the player. We usually tend to over think and over exagerate the problems but players can handle a lot of crap haha

    2- Forget about implementing my own sync system and and go full Photon TrueSync

    The problem: I'm still a bit skeptical about using Lockstep for mobile a mobile game. Connections are getting better with the tame and most ppl have 4g, but still I'm not sure of the impact of this and latencies of 100-200ms.

    Also, I'm not completely sure how would TreSync work in my scenario and if it would solve my problem.
    My only inputs are clicks on the units buttons, and it doesn't happen very often.

    Will TrueSync sync my units positions, spawns, deaths etc. with 100% accuracy for both clients?

    Also, using Spehere Colliders as Triggers for my units attack ranges counts as physics? I guess yes since it's calculating intersection between 2 spheres even if the don't "collide". I could change the Range System without any problem, It's just a radius from the center of the unit.

    Oh, I also forgot to mention that the ammount of concurrent units in screen will average 12 with a peak of 24-28 under very rare circumstances.
  • Hi @MaRTiDoRe, we have a new implementation of TrueSync to be released soon next year, it had some changes in its road map so we could not release it before, maybe it worth to you try an own implementation for now, as you have some code and knowledge about networked game simulation. The TrueSync version in Asset Store could handle with 100% accuracy what you need, but you may suffer some latency problem as it has an old simulation code.