Multiple load balancing apps on one machine

Hello, we use Load Balancing as base for our server applications, for our unity games.
We have a test server, where we host our server app.
We have recently decided to run tye second default load balancing app on the same machine as our main one is hosted, so we wont need to buy second server machine for our tests.
Is that possible? I guess we need ro setup ports that our apps use, and just run PhotonCloudService without PhotonControl?
We have default configs for both apps. Please, help to set that app.
Thanks

Best Answers

  • chvetsov
    chvetsov mod
    Answer ✓
    i think i know what is going on

    In your config for port 4599 add OverrideApplication = Master_New. otherwise new game server connects to this port and sends app name "Master'. without Override application it will be connected to master. this what we see

    best,
    ilya
  • pavelbaslak
    Answer ✓
    @chvetsov boom! That did a trick! Finally, many thanks!

    Everyone who looks for the solution:
    You want Load Balancing app to start more then default Master/Game applications. Here what you need to do:
    1) Go to deploy/LoadBalancing and put there another application that you want to start. In my case, I have copy/pasted Game and Master folder, so they have become Game_New, Master_New
    2) Go to deploy/bin_Win64 and edit PhotonServer.config file.
    3) You should add new values:
    Under LoadBalancing/Applications add new nodes for every app you have added. Ensure you also have set BaseDirectory property to proper value, where your new apps located.
    Then create Udp, Tcp and WebSocket listeners for your new apps, ensure your OverrideApplication property points to proper application
    4) Go to deploy/LoadBalancing/"Name of application that you have added" in my case /Game_New and then /Master_New and open Photon.LoadBalancing.dll.config file for every app added. There you need to change values for Udp, Tcp, WebSocket listeners to the ones you have created at point 3.
    5) Enjoy, you should not have waste X hours on the solution :)) @chvetsov once again Thanks, ExitGames support is excellent!
«1

Answers

  • hi, @pavelbaslak

    probably the easiest way to do it is just copy&past current application config. Give new names for new applications and create listeners, which will be used to establish a connection. Because you will need to update LoadBalancing.dll.config, you need also copy game and master server application to a new directory.
    so, in short:
    1. update PhotonServer.config
    2. Copy application to the new folder and update configs there too

    best,
    ilya
  • Hello @chvetsov, thanks for an answer!
    What should I update in photonserver.config and in another configs to?
  • well, you need to copy & past existing application, give them new names and update location for them.
    then you need to create listeners which will be used by this new applications. you can not use old. then copy&past files on the disc to create the copy of the existing applictions. you will need to update Loadbalncing.dll.config. there are ports. you should set correct ports in this config for master and for game server

    best,
    ilya
  • So:
    1) copy paste existing application
    2) what should I rename? Exe files? Or all loadbalancing occurances?
    3) are listeners listed in config files? It would be just port changes?
    4) that is it, or still would need tk update ports in config?
  • Thanks Ilya! Have a nice day
  • @chvetsov Ilya,
    can you please assist me with follwing:
    I have added 2 new applications to config, and added 2 more listeners for UDP, and set up port for that to 5057, 5058.
    However, when I try to connect to the game, to just newly created server application(Master_New has 5057), I do receive
    Operation failed: OperationResponse 227: ReturnCode: 32762 (Failed to get server instance.). Parameters: {} Server: MasterServer
    If I try connect to my old application that was working on port 5055, I get:
    Receive issue. State: Connected. Server: '127.0.0.1' Exception: System.Net.Sockets.SocketException: remote host has disconnected on first try, and the second time I try to connect it just hangs

    Here is link to my config file, I guess all is setup there.
    https://we.tl/j9uc5Ten56
  • here is where Loadbalancing.dll.config comes to play. you should create extra listener for server2server communication current has port 4520, add one more. and update loadbalancing config to get connected to right port

    best,
    ilya
  • pavelbaslak
    edited May 2018
    @chvetsov added one more node to TCPListeners like that:

    <TCPListener IPAddress="0.0.0.0" Port="4521"> </TCPListener>
    And updated LoadBalancing.dll.config for both new apps (Master_New, Game_New) like
    <setting name="OutgoingMasterServerPeerPort" serializeAs="String">
    <value>4521</value>
    <setting name="IncomingGameServerPeerPort" serializeAs="String">
    <value>4521</value>

    And still get Operation failed: OperationResponse 227: ReturnCode: 32762 (Failed to get server instance.). Parameters: {} Server: MasterServer
    However, I can connect to working server
  • pavelbaslak
    edited May 2018
    @chvetsov
    Have added new TCP Listener:
    <TCPListener
    IPAddress="0.0.0.0"
    Port="4521">
    </TCPListener>
    And updated both LoadBalancing.config for Master_New, Game_New like:
    <setting name="OutgoingMasterServerPeerPort" serializeAs="String">
    <value>4521</value>
    <setting name="IncomingGameServerPeerPort" serializeAs="String">
    <value>4521</value>
    And still get this error:
    Operation failed: OperationResponse 227: ReturnCode: 32762 (Failed to get server instance.). Parameters: {} Server: MasterServer

    However, I can join our main server app well
  • shovancj
    shovancj
    edited May 2018
    after reviewing the following thread it helped me with connecting voice to a separate service in self hosted mode.

    Maybe an issue you are running into is that you duplicated the game and master folders inside of load balancing, instead of duplicating the entire load balancing folder and renaming it. I also renamed the contained game and master.

    Then in the config file i made a separate application path.. so that photon control would show two photons.

    <<Loadbalancing>>
    ...


    ...


    and then changed upped all the ports inside of loadbalancing new.. ie 5055 -> 5057 so on and so forth.

    maybe this will helps some.
  • the xml formating broke in the post above..
    [Loadbalancing]
    ..
    [/Loadbalancing]
    [LoadbalancingNew]
    ..
    [/LoadbalancingNew]

    assume angle brackets
  • @pavelbaslak please check logs first in deploy/log for your second game and master servers and see whether new game server connects to right master and whether new master gets an incoming connection.

    if you will fail to find the reason, then please share with me configs and logs.

    best,
    ilya
  • @chvetsov
    I have noticed that it can connect to Master server, but cant register GameServer, seems like some setting is missing still. I guess it is just 1 step missing or so.
    Here is log
    2018-06-01 10:45:06,243 [11] INFO Photon.LoadBalancing.GameServer.MasterServerConnectionBase - Connecting to master at 127.0.0.1:4521, serverId=d7389ca9-725c-4063-a15b-27a3edde76af
    2018-06-01 10:45:06,244 [30] INFO Photon.LoadBalancing.GameServer.MasterServerConnectionBase - Master connection established: address:127.0.0.1
    2018-06-01 10:45:06,394 [30] INFO Photon.Common.LoadBalancer.ServerStateManager - Server state is set to online
    2018-06-01 10:45:06,452 [30] INFO Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - Registering game server with address 127.0.0.1, TCP 4531, UDP 5056, WebSocket 9091, Secure WebSocket 19091, HTTP 80, ServerID d7389ca9-725c-4063-a15b-27a3edde76af, Hostname localhost, IPv6Address
    2018-06-01 10:45:06,509 [55] ERROR Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - RegisterGameServer returned with err -2: Unknown operation code
    2018-06-01 10:45:06,555 [55] ERROR Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - RegisterGameServerResponse contract invalid: Missing value 2 (RegisterGameServerResponse.InternalAddress)

    2018-06-01 10:45:06,598 [54] INFO Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - connection to master closed (id=160, reason=ManagedDisconnect, detail=), serverId=d7389ca9-725c-4063-a15b-27a3edde76af

    And here is the link to config files and logs
  • from config point of view, I see only one error. Your GameServer_new listens on 5058, but you have 5056 in the configuration file. this port belongs to 'GameServer'

    just set a breakpoint in MasterAppliction.CreatePeer and check why it creates the wrong peer for your game server. that is why you get answer 'Unknown operation code'

    best,
    ilya
  • Hm, changed that value, and still is the same;
    It tells that contract invalid, can this be something with settings still, as I see that contract is being assembled from settings file. It tells missing value 2, is that PublicHostName ?

    Trying to catch CreatePeer, but does not get inside
  • >It tells that contract invalid, can this be something with settings still, as I see that contract is being assembled from settings file. It tells missing value 2, is that PublicHostName ?
    it is invalid because it got non expected response.

    Reason for that is on Master server. if breakpoint does not work, then add logging. i assume that breakpoint does not work because you did not put correct dll/pdb to right folder.

    what version of sdk are you using?

    best,
    ilya
  • pavelbaslak
    edited June 2018
    Hello @chvetsov !
    I have attached the debugger, and have found that in MasterApplication.CreatePeer, I receive initRequest param with probably wrong data in.
    I have GSGame_New.log opened, which one refering to one of my GS_New application with (5057, 5058) ports.
    So, this log is being updated once in X seconds. If I am attached to debugger, and not going further, the log stops, what probably means that Im attached to correct Application.
    However, if I spotlight initRequest parameter, I have following data in(attached a screenshot).
    As you can see, there is LocalPort parameter 5055, while it should probably be 5057, for me second Game Application.

    Image

    @shovancj Thanks for your input, but unfortunately your solution had the same result, as I currently have(
  • hi, @pavelbaslak
    everything works as expected.

    You have attached to late. S2S connections already established. What you see is client connection to your Master server. it is done on port 5055.

    you may start photon right from Visual Studio. Set LoadBalancing as startup project.

    open properties for LoadBalancing: Selected 'Debug'. set 'Start external program' to your PhotonSocketServer.exe location
    Command line arguments: /debug LoadBalancing /config PhotonServer.LoadBalancing-Development.config
    PhotonServer.LoadBalancing-Development.config - is config name. you may replace it to your config name. but you should put your config near this one in LoadBalancing folder.

    Working directory: ..\..


    In such a way you will be able to start photon by pressing F5. Be ready that you will get stopped twice. once for port 4520 and once for 4521.

    also, you may add logging into this piece of code, to see what is going on

    best,
    ilya
  • I never actually tested to make sure that my Voice was still working. The CCU limit dropped to 1 per application and I was satisfied to move forward but now that I have realized that Voice broke in doing so I am back where I started.

    I Get PunVoice: connection to master 127.0.0.1:5057 in the LOG, and no error after. although I also never get my PhotonVoiceNetwork.ClientState to be Joined. 308 VoiceDemoUI.cs.

    The ClientState remains Joining.

    When i switch the photonserversettings->photonvoiceSettings->serverPortUDP back to 5055 it connects fine.. but my CCU increases and Im not using the separate server for voice com traffic.

    Does anyone have a solution for this?

  • hi, @shovancj
    I'm sorry to say that I do not understand what is your problem is.

    If you connect to the server then your connection will be counted, no matter on which port you are connected.

    best,
    ilya
  • Hey! @chvetsov

    Sorry i was directed to this thread from another thread
    https://forum.photonengine.com/discussion/comment/39863#Comment_39863

    It goes over my issue a bit better.

    Mainly Im getting a ccu count for each photon and punvoice on each application running, so essentially 2 ccu hits per application. I was told that connecting seperate server instances on seperate ports provided in the photonserversettings would alleviate this.

    Thanks.
  • Hey @chvetsov
    I have finally returned to it back,
    and now trying to debug that thing, I have setup my project to connect to bin_Win64. You have told, that debug will hit twice, for both Master applications, however, it fires only ones, for main Master application, and after a while for game server that connects to first master, but no call in CreatePeer() for second master application
  • hi, pavel

    Are you sure that a second couple of servers started?

    best,
    ilya
  • @chvetsov So, this time what I tried to do:
    1) Downloaded latest server SDK;
    2) In bin_Win64 PhotonServer.config added new application LoadBalancing_New, edited ports and paths, duplicated Game, Master, so there also are Game_New, Master_New.
    Config files are attached:
    https://we.tl/UbzdDVOFHy

    3) so now if I start Loadbalancing_New from photon control it disconnects. While first LoadBalancing works. The same is with starting project from VS, it just starts LoadBalancing, while _New one is greyed out
  • Hi, @pavelbaslak

    configs look correct. in order to see what went wrong, I need to see logs from bin_Win64. You also should check them to see what is wrong

    best,
    ilya
  • @chvetsov
    it is all like was before:

    2018-07-16 13:33:18,066 [28] INFO Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - connection to master closed (id=22, reason=ManagedDisconnect, detail=), serverId=a9836839-cfa5-441e-bc24-1de3f13c5e0c
    2018-07-16 13:33:33,095 [8] INFO Photon.LoadBalancing.GameServer.MasterServerConnectionBase - Connecting to master at 127.0.0.1:4515, serverId=a9836839-cfa5-441e-bc24-1de3f13c5e0c
    2018-07-16 13:33:33,097 [18] INFO Photon.LoadBalancing.GameServer.MasterServerConnectionBase - Master connection established: address:127.0.0.1
    2018-07-16 13:33:33,130 [18] INFO Photon.Common.LoadBalancer.ServerStateManager - Server state is set to online
    2018-07-16 13:33:33,144 [18] INFO Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - Registering game server with address 127.0.0.1, TCP 4526, UDP 5051, WebSocket 9086, Secure WebSocket 19091, HTTP 80, ServerID a9836839-cfa5-441e-bc24-1de3f13c5e0c, Hostname localhost, IPv6Address
    2018-07-16 13:33:33,159 [18] ERROR Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - RegisterGameServer returned with err -2: Unknown operation code
    2018-07-16 13:33:33,173 [18] ERROR Photon.LoadBalancing.GameServer.OutgoingMasterServerPeer - RegisterGameServerResponse contract invalid: Missing value 2 (RegisterGameServerResponse.InternalAddress)
  • please send me full logs together with logs from bin_win64 folder

    best,
    ilya
  • because you are playing with two configurations it is difficult to see what should i look for and where

    so, i found for 'XXX_new' next error in logs:
    2052: 17:31:43.270 - CService::OnException() - Exception: CSocket::Bind() 127.0.0.1:5050 - An attempt was made to access a socket in a way forbidden by its access permissions.

    I think this is clear. It looks like the second instance of photon socket server started. and you can not get access to socket

    best,
    ilya