Master Server

Curtis37

I have recently been trying to build a server for a lobby based game. I have watched CJR Gaming's Introduction to Photon 3.0 and his Server2Server tutorials. Where I am having my problems is trying to figure out how to design a scalable server framework.

My current understanding is you build a master server (load-balancing server). Both the clients and the sub servers connect to the master server. I would then create a login server (used to validate accounts and store information on the database), multiple game servers (for hosting rooms and game matches), and a chat server (for tells, room and global chat). So it would end up looking like this:

[Clients] -> [Master Server] <- ([Login] [Game Servers] [Chat])

Furthermore I understand that the clients connect to the master server and the master server redirects its requests to the game servers which then bounce back to the respective clients. The client only ever connects the the master server and does not disconnect and reconnect to multiple servers.

*I am trying to figure out a design that will allow for scalability.

Questions:
1. How does the master server not become a bottleneck? Wouldn't routing all the traffic through a single point cause a network bottleneck?
2. If I am then to create an array of master servers how would I tell the client which master server to connect to?

Kaiserludi

Our LoadBalancing app works like the following:
Clients connect to the masterserver and there they then can join the lobby to get lists of open games (if you don't set the according property to false, the clients will do that automatically, everytime they join the master), they can create gamerooms, join certain gamerooms by the name of the room or join rancom gamerooms. Wheneve a client creates or joins a room on the masterserver and no error occurs (like, no random rooms with the specified criteria or no room with the specified name coudl be found), the masterserver will answer with the adress of the gameserver, on which the room to join is open or in case of a create room creation request from the client, the client should create the room (the decision on which gameserver to create a room is done by comparing the current load of all gameservers, so that no one gets more traffic than it can handle). The client then disconnects from the master and connects to the specified gameserver. All ingame network traffic is then purely between the clients in that room and the gameserver. When a client leaves a room, it disconnects from the gameserver and reconnects to the masterserver. The client side logic is all done fully automatically by the client side LoadBalancing implementation, so your application don't need to care about this at all. If you are interested in implementation details, then just have a look at the source. The LoadBalancing layer is completely open source on both, the client- and the serverside. We use the same clientside implementation and nearly the same serverside implementation (for the topic we are talking about, you can assume, its exactly the same) for our cloud and although we have hundreds of different apps connecting to it and some of the top-apps have several thousand clients online at once (and hundredthousands of downloaders of their app), we can still easily handle the whole cloud with just one masterserver (although by now we have quite a few gameservers running in our cloud).

As long as you don't have several millions of customers for your game, you can be sure, that the masterserver won't be a botleneck.

Curtis37

Ok that makes some sense. Originally I assumed the master server worked as a proxy (as it did in the user tutorial). This made sense because then the user can be verified on the master server and I don't have to worry about verification as it doesn't have to disconnect and reconnect to game servers, the information was just routed through the server2server manager.

So my new question is how would I then verify on the game server? Would I first verify the credentials (I am using an SQL database account system) on the master server, and then when they create a match it disconnects, and then connects to the game server. Upon connecting to the game server how will the game server know which in game peer to attach the connecting client to? And then will I have to re-validate again? And then each time it disconnects and reconnects to each server?

Also I am looking at src-server/LoadBalancing/ what is TestClient? And using unity what is the appropriate client to use to connect to the load balancing server. I see in Photon Unity SDK/demo-loadbalancing-unity/Assets/DemoScene.unity it specifies that it connects to the cloud, is this the demo that I use to connect to my local load balancing server? Do I need to just change the connection info?

Kaiserludi

The LoadBalancing app has a build in authentication operation, that will be sent by the client autoamtically on every connect to the master- or a gameserver, and contains the appID and the appVersion (this is sent encrypted on default, but you can deactivate the encryption, if you want to) and the servers won't answer any other request from a client with anything else than error-messages that it has not authenticated, until a client authenticates correctly first, but this only assures, that no one else, that knows their adress can use your server and join your gamerooms with other clients than your ones as long as you keep your appID save, and that no incompatible versions can end up in the same gamerooms (as long as cou count the versionnumber up everytime a clientversion is incompatible with the previous one, however you should not count it up, if its compatible, so that compatible versions can see each other and join the same gamerooms).
If you need to verify user credentials, I would recommend to either inherit from the LoadBalancingClient class on the client side and override the authenticate operation by sending additonal info and adjust the server accordingly to check that info against the database, or to add a new operation theirfor and yes, this info has to be sent after every connect.

The testClient is a basic client, that can be used to check, if the server has been set up correctly, so if your clients can't connect to the server, you can run the testclient to determine, if you are experiencing a server- or a client-side issue. It can alo make sense, to run it from another computer than the one, the server is running on, to check for stuff like firewall issues, if your clients from devices can't connect, but the testclient from a local computer can (for some client plaforms there is no way to run the clients locally on the server, as the server is Windows only and for example iOS has no simulator, that runs on Windows).

For unity you have the choice to use either the Unity client SDK from our downloadarea or the Photon Unity plugin from the unity app store. The letter one mostly matches the API of the Unity builtin networking and is intended for an as quick as possible porting of games, that are already running with the builtin networking, but can't live with the limitations of the builtin networking any longer. For development of the network gamelogic from scratch the normal Unity client SDK is preferable, as its API is far more flexible for custom needs, especially if you want to have custom serverside logic, like with your SQL database account system. So, you are already looking into the appropriate client. You would go to Assets/DemoCode/DemoGUI.cs and change line 15 this.GameInstance.MasterServerAddress = "app.exitgamescloud.com:5055"; (talking about Photon-Unity3D_v3-0-1-12_SDK here, the line number and content can differ for different client versions and will differ for different client platfroms) to the address of your masterserver (5055 is the default port, only change that part, if you have changed that port in the server config). Thats the only client side change you have to make. After you have set up and started your server, you are ready to go.

Curtis37

Ok so just to clarify the client would connect to the load balancing server, a peer would be created. The client would then get login credentials from the user and store these globally (encrypted) for later connections. These credentials get sent to the load balancing server which will then unlock the further functions like creating matches. Then once I create a match the master (load balancing) server would tell the game server to make a room and a peer/actor(or some object used for that user) would be created and be setup on that server in a state waiting for a connection. The master server then tells the client what game server to connect to and then disconnects from the client.

The client then connects to the game server using the information given from the master server. Once the client connects to the game server a peer would be created. The client sends its stored information to the game server. The game server would then do a lookup and try to find a matching object given the credentials provided. It could then test the credentials and if there is a match that peer connection would be attached to the appropriate object.

The parts I am worried about is storing the credentials on the client, I imagine this could be hazardous but I am not experienced to say why or if it is that big of a deal. I don't know what ability hackers would have to retrieve this information (since it would be secure over the internet) the only apparent way I could assume would be to memory scan the users computer and idk if this is a worry at all. Also I think it would be annoying if the client had to manually type their info each time, which I assume could also be picked up by key loggers so I guess that is assumed to not be happening.

Is there anything further security wise that I am missing? Or any further Photon tools that can help me keep this handshake secure?

Kaiserludi

The master server doesn't tell the game server to create a room and does not disconnect the client. It just chooses a game server and passes its ip to the client. The client then disconnects from the master and connects to the game server, for which it has received the address. After connecting to it, the game server assigns a peer to the client, under which it will recognize it for the time of that connection and the client then authenticates itself on the game server. After a successful authentication the game server will allow this peer further actions. Now the client creates or joins a room on the game server. In case of creation of a new room the client sends a request to the game server to create it and now the game server will create the room. The game server also informs the master about all clients connecting to and disconnecting from it and about open rooms, so that the master can account for those clients when deciding where to open the next room or where it has to send clients to, that want to join a certain room. The game server also informs the master on which rooms are open and / or visible and which ones have reached their max player limit, which has been specified by the client, which has sent the creation request, and of course, about any room properties, that should be listed in the lobby and that should be honored for random matchmaking on certain criteria.

I would not ask the player to input his/her credentials every time, (s)he is joining a game, again, as that would be quite annoying for him/her, but store it in a client side variable (not in a file on the disk, its perfectly fine to ask the user for the credentials again after a restart of the app). If you are worried about security, then just store it encrypted. It would need some serious knowledge and work to identify the location in RAM, that is holding the credentials, while the game is running, so its nothing a normal user could do, but sure, its possible, so it makes sense, to store this information encrypted. Don't reinvent the wheel here, but use an existing encryption algorithm, that has been thoroughly tested by a lot of people, to not be beatable in a reasonable time on reasonable hardware.

Curtis37

Ok, I think everything makes sense now. Thanks a lot.

Edit:
One extra problem. I am running "Photon Unity SDK/demo-loadbalancing-unity/", I changed

DemoGUI.cs
this.GameInstance.MasterServerAddress = "127.0.0.1:5055";
this.GameInstance.ConnectToMaster("Master", "1.0", "unityPlayer");

So it no longer tries to connect to the cloud. I then ran the "Load Balancing (Cloud)" Photon instance (the one with "Master" "Game Server 1" and "Game Server 2"). It connects fine and when I hit create game it appears to work fine as well. But then when in the game I am presented with the options to move and change game parameters. When I click move it doesn't seem to function and when running the test client to populate the game servers the test client displays "error in event response" 4 times. My player name also displays as "unityPlayerunityPlayer73" which seems weird and custom props is showing {(String)0=(String)0}.

Question:
So are these two projects supposed to match up? ("Photon Unity SDK/demo-loadbalancing-unity/") and (Photon Server 3.0/deploy/Loadbalancing)?

[Deleted User]

Hi Curtis37,

this sounds like your lokal server config is somehow broken. According to your description, I tested this demo code with:

this.GameInstance.MasterServerAddress = "192.168.78.188:5055";
this.GameInstance.ConnectToMaster("<insert your app id here>", "1.0", "unityPlayer");

All I did, was activating 'Loadbalancing (MyCloud) via Photon control and setting IP config to the one setup in 'this.GameInstance.MasterServerAddress'. To actually see the sent data (when pressing 'Set Room Property'), I altered the buttons code to:

if (GUILayout.Button("Set Room Property"))
{
   Hashtable randomProps = new Hashtable();
   randomProps["data"] = "Testdata" + Random.Range(0, 100).ToString();
   this.GameInstance.CurrentRoom.SetCustomProperties(randomProps);
}

and matching this, I catched it via:

case EventCode.PropertiesChanged:
   var data = photonEvent.Parameters[ParameterCode.Properties] as Hashtable;
   DebugReturn(DebugLevel.ALL, "got something: " + (data["data"] as string));
   break;

in 'public override void OnEvent(EventData photonEvent)'. Having this done, you should be able to see, that:
1. (to your question): These projects are compatible.
2. The received data is correct.

If you played around with your servers config, I would suggest, you simply reinstall the server, if thats an option for you.


Hope that helps



Tim

Curtis37

I have been reading the documentation (docs/lite & lite lobby). It doesn't explain too much but it mostly makes sense. Where I am confused is understanding the LoadBalancing VS solution. It contains 3 projects the Loadbalancing.TestClient is straight forward but when it comes to Loadbalancing and Lite how do they contribute to the game server and master server portions?

Is Lite = game server, and LoadBalancing = master server? Or do they equally contribute? Also by the way I am using deploy.prompter.cmd to compile the package.

Lastly, is there an API anywhere? Like this (http://docs.oracle.com/javase/6/docs/api/)? It would be nice to see some in depth information about each package and maybe an idea of the class structure and hierarchy if that exists?

[Deleted User]

Hi, we have just recently added some LoadBalancing documentation to our online help: http://doc.exitgames.com/photon-server/ ... references - it describes the "big picture", and also the structure of the VS Solution. I recommend to have the solution opened and browse through the implementation while you read the documentation.

Basically, GameServer = Lite (the GS app inherits from Lite, with only little extensions), and the Master server is an application built "from scratch", it is not inherited from Lite.

Regarding an API documentation: it exists only for the "base classes" in the Photon Server SDK in /doc/Photon.SocketServer.chm, and for MMO in /doc/applications/Photon.MMoDemo.chm.
I'll see if we can add API documentation for the remaining applications in a future Photon Server SDK release.

Curtis37

Right now my main problems lie trying to fully understand the contents of Photon3Unity3D.dll. Could you shed some light on the purpose of PhotonPeer.Service()

Also where does the client send a request to authenticate. In the LoadBalancingFlowchart @ http://doc.exitgames.com/photon-server/LoadBalancing/#cat-references it shows that the Client sends an authentication request and the master server responds. I am trying to work with the authenticate to require it to send a username and a password. I am a little confused how the client is sending this request.

EDIT: It looks like I have figured it out for the most part.