Dashboard data issue

KevinB

Hi,

when I look at the dashboard real time data, once in a while data will be missing. see the pic below where data is missing at time around 14:40:30 and 14:42:00. These data are published by the game app. I know that the server isn't down because data published by the counter publisher from the same server still shows that we have connected peer during this period.




I am a little worried about data missing for almost a minute by the game app. we have been experiencing disconnect issue by our clients, not sure if anything related.

have you guys seen anything like this before? and what could possibly cause this? Thanks.

K

KevinB

this actually happens quite frequent.

K

[Deleted User]

Hi Kevin,

it's a bit hart to tell what is going on, I can only share some ideas that should help to analyse the issue better.

Photon publishes data to the dashboard service per UDP. UDP is unreliable - so if your network or CPU is overloaded, packages might be dropped, which could cause the gap in your graphs. In this case, there might be a correlation to client disconnects.

You say that it happens frequently - this gives us a chance for debugging. You can try this:

1.) have a look at the log file in \deploy\bin_tools\dashboard\log\log.txt

The Photon apps are publishing data every 10 seconds, each from a separate socket. If you have 4 apps in Photon that publish data, the log entries will be repeated every 4 lines, like this:

[code2=plain]2013-08-20 15:46:13,511 [8] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.169:63876.
2013-08-20 15:46:13,823 [8] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.169:63878.
2013-08-20 15:46:14,134 [6] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.169:63879.
2013-08-20 15:46:15,167 [6] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.120:50250.

2013-08-20 15:46:23,517 [6] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.169:63876.
2013-08-20 15:46:23,822 [6] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.169:63878.
2013-08-20 15:46:24,139 [6] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.169:63879.
2013-08-20 15:46:25,177 [6] DEBUG ExitGames.Diagnostics.Monitoring.CounterSampleReceiver - Received data from 192.168.78.120:50250.[/code2]

Is there a gap in the received data as well? (e.g., one of the entries is not repeated as expected, is delayed or missing at all).

2. check the SocketServer.* and System.* images in your dashboards. Are there peaks for network traffic, CPU usage, disconnects? Anything unusual?

3. The dashboard data relies on the Windows Performance counters, which give more precise information.
You can collect a set of performance counters, and send me the data for an analysis when you notice a gap in your dashboard data again. These are the required steps (the order is important!):

[code2=plain]The order of these steps is important!
1.) stop Photon, if it's running
2.) from Photon Control -> Performance counters - Install counters, - Create logging set - Open perfmon
3.) In PerfMon: under "data collector sets" -> "user defined": choose the photon_perf_log in the RIGHT window pane -> properties -> set the sample interval to 1 second
(the default of 1 minute does not give us enough data here)
4.) start photon
5.) photon control -> performance counters -> start logging
6.) let it run until you see the gap in your dashboard data. Attention - a counter log file can reach several GB per day!
7.) photon control -> performance counters -> stop logging
8.) Get performance logs from C:\perflogs\admin, zip them up and put them somewhere where I can download them.[/code2]