[Unity] Server time on same machine that's running server

Seigman

I've been smashing my head against the wall figuring out why my interpolation code wasn't working properly between actors. I've overloaded the LocalMsTimestampDelegate in the PhotonPeer with a function that returns the UnityEngine.Time.time. This improved the timestamp accuracy and made my interpolation very smooth.

However, when I run an instance of my Unity project on the same machine that's running the photon server, and then connect from a second machine the timestamp that the second machine receives are ~300ms off. This is not due to network lag or anything (running on a local lan, and I've monitored the server through the dashboard). This happens with all connected actors, except for the one connected on the same machine that's running the server.

Also, if I run several instances of the Unity project on the same machine that's running the server, only one of the instances get a proper timestamp, while the rest lag behind ~300ms.

Now, if I instead connect from two or more machines and don't connect from the machine that's running the server, all timestamps are in sync and everything is well. As the documentation about this area is a bit vague (what exactly does the LocalMsTimestampDelegate actually do?) I'm wondering if anyone has any insight as to why this is?

Strange thing is that I just use the peer.ServerTimeInMilliSeconds to timestamp each package in the unity project, and use the same value in my interpolation code (based on the Unity networking example).

For deployment this wont be an issue as the server will be running on a standalone machine, but for development I don't want to have to use several machines to connect.

Tobias

The LocalMsTimestampDelegate uses Environment.TickCount to get the current timestamp. Environment.TickCount is not very precise but not as far off as 300ms. The delegate is there so you can override the time-getting method, exactly as you do now.

I will have to check if I can reproduce the 300ms lag somehow. How do you measure this?

I tested with the DotNet Realtime Demo and the server time approximation was ok on a local machine. I think I also used a few clients from a different machine but in most cases, I admit, I has some client on the server's machine as well.

Seigman

Overriding the LocalMsTimestampDelegate seems to give me a more precise ServerMsTimeStamp, as that's the value I use for timestamping the packages. Could you explain why this is? If I had the source for the backend system I'm sure I'd figure out my problem, but as I dont, and the documentation and examples are rather lacking I'm sort of in the dark here.

I'm taking the incoming timestamp (ServerTimeMsTimeStamp from another client) and then subtracting it from the ServerTimeMsTimeStamp on the client. On one of the clients this is around the values I'd expect (50ms), but on everyone else it lies around 380-420ms... As I have no idea of what's going on behind the scenes, I'm not sure if the issue is my usage of Photon (a proper example would have helped here), or if there's an issue in the backend system (source would help here), I'm starting to run out of ideas of what the possible cause for this issue might be...?

Seigman

Turns out this was completely my fault...

To begin with I tried calling UnityEngine.Time.time directly in the LocalMsTimestampDelegate, but as Photon runs in a separate thread I was getting an exception about not being able to call get_time() outside of the main thread. My solution to this was to cache the UnityEngine.Time.time value into a variable that I fetched into on every Update() and FixedUpdate() call in Unity.

Now, in my Unity project I'm loading between scenes (game lobby first, then load the game scene before starting the game). As the game scenes are somewhat big it took some time to load on some machines. The PhotonPeer is set to send a ping to the server every 1000ms to keep the connection alive if it hasn't sent any packages lately. As Photon runs in a separate thread it was doing this while I was loading levels in Unity, and like I said before, I was caching the value for my local time on every Update(), but they weren't being called while loading levels so the longer it took to load a level, the more the client would become offsync with the server.

My solution to this was to increase the TimePingInterval on the PhotonPeer before loading level to avoid it doing any pings with an old time value, but perhaps there is a better solution to this? Maybe it's even possible to grab the UnityEngine.Time.time value directly from the delegate?

Tobias

Puh, I'm glad to hear you found a cause for this. I didn't have a chance to look into it but I by the description I wasn't sure where we failed...

By the way: We already take care of pings. You don't need to do that! If you don't send anything reliable in a while, a ping is generated, sent and ackd. This keeps the connection alive.
You only have to take care that you call Service() or SendOutgoingCommands() a few times a second (or at least once a second).

Then: I'm not enough into Unity at the moment to see why you can't call UnityEngine.Time.time in some other thread. But why do you run Photon in another thread at all?

We made the client lib run in a single thread and non-blocking to allow you to incorporate it into your own game loop. You can call Service() in Update() and the good thing (I thought) is that all callbacks are triggered by Service(). It means: when you reach the point in your loop where you call Service(), you are sure that nothing else is accessing the player/item/game data you update by calling Service() and in turn EventAction().
If you "outsource" Service() to another thread, you need to take care when you get events that you don't access game's data while the gui or someone else accesses the same data?...

To access the delegate's return, use LocalTimeInMilliSeconds:
public int LocalTimeInMilliSeconds
{
get { return this.peerBase.GetLocalMsTimestamp(); }
}

Seigman

Have you tried calling UnityEngine.Time.time directly in a LocalTimeStampDelegate? The delegate is called from the thread you're running Photon on, and the time property in Unity causes an exception if it's called outside of the main thread. That's why I fetched the time into a variable in the Update callback from Unity and then used that to return a local time in the delegate.

Problem is that while a level is being loaded in Unity there is no Update callbacks, and therefor the cached time value isn't updated leaving Photon to grab an outdated time value when Unity is loading. That's why I increased the time between each ping call from the PhotonPeer before I start loading a scene (and then revert it back to default value when level has been loaded). I'm also pausing any calls to Service during the level load...

Seigman

Callstack for exception:

UnityEngine.Time:get_time()
CSyncManager:GetLocalMsTimestamp() (at Assets\Plugins\CSync\CSyncManager.cs:425)
ExitGames.Client.Photon.EnetPeer:receiveIncomingCommands(Byte[])
ExitGames.Client.Photon.NConnect:Run()

Tobias

"I'm also pausing any calls to Service during the level load..."
That will cause timeout-disconnects if your load times increase.

I would do most work in the Update. Especially your own pinging and calls to Service. As you don't get Update calls while loading, you could use a separate thread which only calls SendOutgoingCommands(). This "preventTimeoutThread" should only do anything during load times (and other stuff that keeps Unity from calling Update).

How does that sound?

Seigman

I know it will cause timeout-disconnects if load time increases, but I can manage that. See the callstack for the exception in my previous post as it seems that the delegate isn't called just from the thread you call Service() from, but also from the Photon thread running in the background...

Seigman

And I'm doing all my work in Update(), it's Photon that's running a thread in the plugin and calling the delegate from that thread which is causing issues for me.

Tobias

Argh. You're right.
I forgot that the asynchronous receive methods are also using it. They have to know the current time to mark time of arrival for incoming commands and this should happen asap after receiving the udp package. I am using the time delegate there to stay consistent with the time values the game uses everywhere else.

I think it doesn't make sense to have the delegate to get better precision for the (server)timestamp if we have to wait for a thread switch after a package was received to calculate it's roundtrip time, cause we can't use the delegate.

Is there an alternative to UnityEngine.Time.time that is open to access from other threads? Maybe Stopwatch.ElapsedMilliseconds is an alternative for you.

Seigman

I was thinking about using Stopwatch as well, just wanted to make sure it wasn't some silly thing I did wrong... Will give Stopwatch a go on monday and let you know.