server SDK & concurrency 101?

mindlube

Hi all sorry for the long-ish post...
Just for background: I am wanting to mod the Photon3 Server SDK to support my turn-based dominos game. Multiple simultaneous matches are allowed & encouraged. So it's really *not* room-based because the matches will persist even with zero active players. Furthermore each Peer will get a stream of updates about it's various "subscribed" matches. Each peer will publish updates about the match id when it takes a turn, or when it joins or leaves a match. So the player is not in a room, in the sense of a Photon room, it's just subscribed to list of match ids.

So I've read through the source for the Lite, LiteLobby and MMo example applications. What I want to do is really even simpler than the Lite application in the sense there are no rooms... but more complex because I need to have some global data structures about all matches and the player's subscriptions and rankings.

What I am looking for are some basic thread safety concepts and examples about how to access persistent storage and shared datastructures.

There are so many concurrency interfaces in the API it's kind of bewildering. Fibers, Threads, Pools, etc.

Is PeerBase.OperationRequest taking place on all 1 thread/fiber/coroutine, or is each connection spawning new thread?

I need to maintain at least these datastructures:
hashtable of interested match ids for each Peer
hashtable of game state for each match id.
hashtable of hiscore for each Player

Would it be safe to just add some static member variables on my PeerBase subclass and assume they are thread safe? Or maybe should I add non-static member vars to my BaseApplication subclass?

Are there any server SDK examples you can point me at that show access to persistent storage, database, leaderboard, or really any kind of global data structures? In a thread safe manner?

Should I be looking more along the lines of the mmo example, and remove the Actor/Item/InterestRegion stuff?

Thanks!

valholtz

Photon does not create new thread for each Peers, Photon creates an AppDomain per application and the AppDomain is non-threaded. For a multiplayer application you'd want threading. However, Photon automatically creates a Fiber called RequestFiber(Fibers are basically threads but can execute in parallel with other fibers rather than threads which basically divides execution time between threads). You can use this fiber to queue tasks. Also, you can create external threads like PoolFiber which uses ThreadPool.

Btw, threading takes a little effort if you want complete thread-safe application. However, if you do not want to go into threading, you can use Fibers combined with ActionQueue, this basically lets you queue tasks in the pool and executes one by one, so Two threads won't access the same resource at the same time(assuming everything that access that particular resource is within the queue).

mindlube

Thanks valholtz, what you wrote mostly makes sense, however you are speaking very broadly. And the semantics are hard to understand for a newb like me. "Fibers are like threads, but not really, and sometimes they are backed by a thread pool... " It's kind of circular and with special cases. From what I've read about Fibers is they are more akin to coroutines than threads. That much I understand, coming from Unity-land, I use C# coroutines all the time....

But I guess I really need to see more example code of server mods. Really there is only 3 provided (Lite, LiteLobby and Mmo). I understand those are the most common use cases, but the Photon API is clearly a lot richer and more flexible than that. Is there a wiki or or someplace where other examples of modding the server SDK are available?

btw I also evaluated Electroserver, but am preferring Photon in terms of it's API and overall design & license cost too. But one area where electroserver was better is in a multitude of different server sdk examples. One electroserver example that was particularly helpful was a server mod that showed how to access a pool of database connections in a thread safe manner. Wishing for similar example(s) for photon!

dreamora

No additional wiki I fear.
The only documentation on that is the one you can find in the CHM.

I guess there is also only limited interest in anything like that as a vast majority of applications easily can go with the Lite - MMO - LoadBalancing instances and addition of their operations into one of these frameworks instead of writing a backend from ground up (which is why most users likely ended here, cause they didn't want or simply couldn't write a backend server code from ground up - I personally cause I don't want to reinvent the wheel a XXth time). As ExitGames staff and other users in the past mentioned, these types of applications are what most users are interested in, there are no other types of games people are really interested in that differ significantly from either of these at the end of the day so its easier to use one of them as base to extend or modify.


valholtz point might seem 'broad' but in the end its exactly like that. If you keep a single fiber per user / room / encapsulated environment and do the communication between them through message / actionqueue then photon will handle all the concurrency for you. Its the core aspect in photons scaleability cause direct usage of Threads is not scaleable, no matter how little the CCU number might be.
The comparision to coroutines here fails completely as unity has no concept of concurrency at all (everything runs in the main thread together with physics and scene graph handling). Fibers on the other hand run on different threads, the only thing thats granted is that the stuff you enqueue on the fiber is executed in a serial form, never in a parallel, hence you can access the data of the related object without fearing concurrent access related problems and without the need of locking which would kill the server performance (thats what I meant with not scaleable above)

mindlube

Thanks dreamora, and valholtz! It definitely helped me to have it paraphrased, explained differently. Yes, I am reading the CHM help file too.
I definitely feel like Photon has all the tools I need and I'm on the verge of banging out a near-perfect solution for my game.
I am noticing how in the Mmo example, the fibers are on the Actor, instead of on the Room, like in the Lite example. I think I'll end up doing someting similar to that. Cheers

dreamora

Actually in Lite you have fibers for actors and rooms as both need these functionality.
In MMO you have it too, just that there is no 'static group' defined called 'Room', its a more flexible and movable object which you likely already know, thats the InterestArea which uses fibers too.
Generally anything that processes information in any form and needs to exist in a scaleable form (more than a singular entity in a single thread) runs on fibers in Photon, at least I don't recall anything that doesn't do it in one or the other form

mindlube

dreamora wrote:

If you keep a single fiber per user / room / encapsulated environment and do the communication between them through message / actionqueue then photon will handle all the concurrency for you

OK just a sanity check (for myself). In my non-room-based custom server app, I have my peer subclass CD3Peer, and my ApplicationBase subclass, App. The App has pool fiber named ExecutionFiber.
In my peer I've simply got the constructor and 2 overrides. I know it's simplistic and not massively scalable but that's OK :

   public CD3Peer(IRpcProtocol rpcProtocol, IPhotonPeer nativePeer)
			: base(rpcProtocol, nativePeer)
        {
        }
        protected override void OnDisconnect(DisconnectReason reasonCode, string reasonDetail)
        {
			App.executionFiber.Enqueue( () => App.PeerDisconnected(this) );
		}	
protected override void OnOperationRequest(OperationRequest operationRequest, SendParameters sendParameters)
        {			
			App.executionFiber.Enqueue( () => App.ExecuteOperation( this, operationRequest, sendParameters ) );
        }

This use of Enqueue() from peer fiber to App fiber is "thread-safe" , correct? Because it's pretty much *identical* to what is going in Lite:
LitePeer -> HandleGameOperation() -> Room.EnqueueOperation() -> this.ExcecutionFiber.Enqueue( () => ... )
I thought I read otherwise, somewhere in the forums, so just checking.
thanks