Some help needed

rokawar

Hello everybody,

Since i have upgraded PUN to the version "1.24" and the server to the version : 3.4.1.2521, when i get around 300 players on my self hosted server , we receive an error on the client when we try to create a room.

createGame failed, client stays on masterserver: OperationResponse 227: ReturnCode: 32762 (Failed to get server instance.). Parameters: {}.

And the UC go up to 70 %

The server has been started yesterday at 10PM and i got this problem today at 11AM.

Before the update everything was fine on my server and it was able to handle more that 1500 players without any problem.

I have try to find some error on the log's file but without result.

The only thing that i have found is on the file "MSMaster.LOG", sometime i got :

2014-04-07 02:37:41,614 [20] DEBUG Photon.LoadBalancing.MasterServer.MasterClientPeer [(null)] - Disconnect: pid=8331: reason=TimeoutDisconnect, detail=
0 - 6 - 0 - 0 - 810786509 - 810842357 - 55848
1 - 16 - 810813981 - 6443
2 - 15 - 810813981 - 7597
3 - 14 - 810813981 - 8799
4 - 13 - 810813981 - 10359
5 - 12 - 810813981 - 13713
6 - 11 - 810813981 - 15538
7 - 10 - 810813981 - 17597
8 - 9 - 810813981 - 20702
9 - 8 - 810813981 - 23385
10 - 7 - 810813981 - 25725
11 - 6 - 810813981 - 27584
12 - 11 - 810799691 - 1724
13 - 10 - 810799691 - 3307
14 - 9 - 810799691 - 6412
15 - 8 - 810799691 - 9095
16 - 7 - 810799691 - 11435
17 - 6 - 810799691 - 13792
18 - 8 - 810792780 - 2184
19 - 7 - 810792780 - 4524
20 - 6 - 810792780 - 6896
21 - 7 - 810789130 - 1724
22 - 6 - 810789130 - 3448
23 - 6 - 810787383 - 1724
24 - 3 - 242 - 155 - 810786369 - 810786478 - 109
25 - 2 - 261 - 162 - 810785230 - 810785355 - 125
26 - 1 - 280 - 170 - 810784107 - 810784216 - 109
27 - 5 - 304 - 170 - 810782952 - 810783093 - 141
28 - 4 - 327 - 173 - 810782952 - 810783093 - 141
29 - 3 - 353 - 169 - 810782796 - 810782921 - 125
30 - 2 - 385 - 149 - 810782609 - 810782781 - 172
31 - 1 - 415 - 128 - 810782422 - 810782578 - 156
32 - 0 - 452 - 85 - 810782297 - 810782406 - 109
33 - 0 - 500 - 0 - 0 - 0 - 0

If you can tell me why i get that since the update and how i can track the issue.

For information we don't have any problem when we connect to the photon cloud server.

Have fun

Roka.

rokawar

I continue to say that something is wrong with the release of your version 3.4.1.2521.

With the same client :

- When i connect all my players to your cloud,everything work perfect.
- When i connect all my players to my self hosted, at 300 players, the PhotonSocketServer.exe go up at 95% and at the room creation we get :

createGame failed, client stays on masterserver: OperationResponse 227: ReturnCode: 32762 (Failed to get server instance.). Parameters: {}.

On the last version, i have never see my cpu at 95% and never got this problem(with more than 1500 players on it).

I don't do any code change, i just start the loadbalancing with exactly the same parameter as before.

Everything work , untill 200/300 players are online and playing.

I have already deleted all old version and restarted the server, but always the same issue.

Maybe someone have updated the photon server and tell to us if he got a problem or no?.

[Deleted User]

Hey roka,

sorry for the late response here.

The "failed to get server instance" error is related to our loadbalancing algorithm. In short - each game server determins it's own work load (which consists of various factors - like CPU usage, bandwidth usage etc.), and after the work load reaches a certain threshold, the game server notifys the master server that it can not handle any more connections. In that case, the master server returns the "failed to get server instance" error when you call CreateGame.

This makes sense when you say you have 95% CPU load. These errors are a symptom of the "high CPU usage", for which we need to find the root cause.

1. Could you probably attach a profiler (like the Profiler from Visual Studio) and see if it shows which methods are causing the high CPU load?
2. Could you send me the log files (Master.log, GSGame.log), please?
3. Could you also collect some performance counters and send them to us?
-> open Photon Control
-> Stop Photon (if it is running)
-> Perfmon Counters: Install Performance Counters
-> Perfmon Counters: Create Logging Set
-> Perfmon Counters: Start Logging
-> Start Photon
-> run your test
-> Perfmon Counters: Stop Logging

Then go to c:\perflogs\Username and send me the counter log file (or upload it and send me a link), please. That would help us a lot to review your issue.

Thanks!


Can you also collect a performance counter log and

rokawar

Thank you for this great answer.

I'm sorry but i have disabled my server (because too many players have made a report about the problem on it) and i have downgraded the photon client/server in order to recover my self hosting. This will be done tomorrow morning.

I have made a backup of the Master.log, GSGame.log before restoring to the old version. I will send these file soon by private message.

That all that i can do at the moment, my first step is to return at one stable version.

I will come back to you after some week when i will make a new test.

Thank again.

[Deleted User]

After we've reviewed some log files, we have probably identified the root cause for the "high CPU usage" issue with Photon Server SDK 3.4. So, for anybody who has the same issue:

This is very likely caused by too much logging.

By mistake, we have released the latest SDK 3.4 with "DEBUG" logging enabled. In the previuos version 3.2, only "INFO" log messages were enabled. So we have a lot more logging now - which can significantly slow down a machine. Especially for high numbers of connections.

A solution would be to edit all "log4net.config" files and replace all occurrences of "DEBUG" with "INFO".

The log settings will also be fixed in the next hotfix release of the Photon Server SDK, which should be released some time in the next 2-3 weeks.