Production Server Crashed - Running Version 2.4.7

JoshF47 · December 2011

I'm running server version 2.4.7 in production (32 bit Windows 2008 Server), and just had it crash completely unexpectedly. The XML below is what I see in the event viewer for this "Application Error". Is there anything more I can provide that would help identify what happened and how to prevent it in the future? Is this possibly something fixed in version 2.6.1?

It appears to have just suddenly crashed for no reason at all. I have now changed the service to restart if it fails again, but as they say, failure is not an option. It of course needs to always run without crashing. Perhaps there is something I have in my code causing this (what, I can't imagine yet - only thing I see happening last in my code is a login completed which does update MySQL database), but I can't think of a single way to know what happened or how to find out more information if it happens again.

Any suggestions on getting more information on what happened either now or the next time it happens and any input on whether this may be a known issue of any kind and an upgrade to 2.6.1 would resolve it?

Thanks,
Josh

The only thing of note in the instance log file is this:

3100: 03:05:00.310 - Purecall - Crash dump creation failed

Event Data:

&lt;Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"&gt;
&lt;System&gt;
&lt;Provider Name="Application Error" /&gt;
&lt;EventID Qualifiers="0"&gt;1000&lt;/EventID&gt;
&lt;Level&gt;2&lt;/Level&gt;
&lt;Task&gt;100&lt;/Task&gt;
&lt;Keywords&gt;0x80000000000000&lt;/Keywords&gt;
&lt;TimeCreated SystemTime="2011-12-15T03:05:00.000Z" /&gt;
&lt;EventRecordID&gt;36298&lt;/EventRecordID&gt;
&lt;Channel&gt;Application&lt;/Channel&gt;
&lt;Computer&gt;ip-XXXXXX7B&lt;/Computer&gt;
&lt;Security /&gt;
&lt;/System&gt;
&lt;EventData&gt;
&lt;Data&gt;PhotonSocketServer.exe&lt;/Data&gt;
&lt;Data&gt;0.0.0.0&lt;/Data&gt;
&lt;Data&gt;4d5a614e&lt;/Data&gt;
&lt;Data&gt;PhotonSocketServer.exe&lt;/Data&gt;
&lt;Data&gt;0.0.0.0&lt;/Data&gt;
&lt;Data&gt;4d5a614e&lt;/Data&gt;
&lt;Data&gt;40000015&lt;/Data&gt;
&lt;Data&gt;0010e242&lt;/Data&gt;
&lt;Data&gt;408&lt;/Data&gt;
&lt;Data&gt;01ccb65fd16e8086&lt;/Data&gt;
&lt;/EventData&gt;
&lt;/Event&gt;


&lt;Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"&gt;
&lt;System&gt;
&lt;Provider Name="Service Control Manager" Guid="{555908D1-A6D7-4695-8E1E-26931D2012F4}" EventSourceName="Service Control Manager" /&gt;
&lt;EventID Qualifiers="49152"&gt;7034&lt;/EventID&gt;
&lt;Version&gt;0&lt;/Version&gt;
&lt;Level&gt;2&lt;/Level&gt;
&lt;Task&gt;0&lt;/Task&gt;
&lt;Opcode&gt;0&lt;/Opcode&gt;
&lt;Keywords&gt;0x80000000000000&lt;/Keywords&gt;
&lt;TimeCreated SystemTime="2011-12-15T03:07:13.000Z" /&gt;
&lt;EventRecordID&gt;59501&lt;/EventRecordID&gt;
&lt;Correlation /&gt;
&lt;Execution ProcessID="0" ThreadID="0" /&gt;
&lt;Channel&gt;System&lt;/Channel&gt;
&lt;Computer&gt;ip-XXXXXX7B&lt;/Computer&gt;
&lt;Security /&gt;
&lt;/System&gt;
&lt;EventData&gt;
&lt;Data Name="param1"&gt;Photon Socket Server: Instance1&lt;/Data&gt;
&lt;Data Name="param2"&gt;1&lt;/Data&gt;
&lt;/EventData&gt;
&lt;/Event&gt;

Tobias · December 2011

Hello Josh,
This looks like a category of issues that we solved over time.
I will have to investigate if 2.6.x has this fixed.

JoshF47 · December 2011

Thanks Tobias, appreciate the quick response and investigation. We will work on upgrading to 2.6.11 as the latest, but it will take us a little while to get it all up and tested, so hoping it will solve it.

Tobias wrote:

Hello Josh,
This looks like a category of issues that we solved over time.
I will have to investigate if 2.6.x has this fixed.

Tobias · December 2011

Sorry for the late follow up.
Not knowing exactly the cause for your crash, we can't be fully sure but we found at least 2 changes that might be related. So, the last v2 version is most likely fixed.
If not, we expect it to be able to write dump files, which wasn't possible before, apparently. This would help in worst case.

How far are you with the upgrade? I hope you didn't run in major roadblocks?

JoshF47 · December 2011

Tobias wrote:

... How far are you with the upgrade? I hope you didn't run in major roadblocks?

We were able to update without much of an issue (our code is extended from the examples), so hopefully we will have time to finish the testing in the next week or so and go live with it. We are kind of anal about testing, and so we take our time particularly with server upgrades.

Thanks,
Josh

JoshF47 · January 2012

Well, we updated to the very latest server version 2-6-11-1647, and just yesterday/early this morning it crashed again. The results in the event data and logs are basically the same, but I'll include them again below just for reference.

Event Data:

&lt;Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"&gt;
&lt;System&gt;
  &lt;Provider Name="Application Error" /&gt; 
  &lt;EventID Qualifiers="0"&gt;1000&lt;/EventID&gt; 
  &lt;Level&gt;2&lt;/Level&gt; 
  &lt;Task&gt;100&lt;/Task&gt; 
  &lt;Keywords&gt;0x80000000000000&lt;/Keywords&gt; 
  &lt;TimeCreated SystemTime="2012-01-28T05:13:14.000Z" /&gt; 
  &lt;EventRecordID&gt;291979&lt;/EventRecordID&gt; 
  &lt;Channel&gt;Application&lt;/Channel&gt; 
  &lt;Computer&gt;ip-XXXXXXX&lt;/Computer&gt; 
  &lt;Security /&gt; 
&lt;/System&gt;
&lt;EventData&gt;
  &lt;Data&gt;PhotonSocketServer.exe&lt;/Data&gt; 
  &lt;Data&gt;2.6.7.525&lt;/Data&gt; 
  &lt;Data&gt;4dde7592&lt;/Data&gt; 
  &lt;Data&gt;PhotonSocketServer.exe&lt;/Data&gt; 
  &lt;Data&gt;2.6.7.525&lt;/Data&gt; 
  &lt;Data&gt;4dde7592&lt;/Data&gt; 
  &lt;Data&gt;40000015&lt;/Data&gt; 
  &lt;Data&gt;00115402&lt;/Data&gt; 
  &lt;Data&gt;9cc&lt;/Data&gt; 
  &lt;Data&gt;01cccadd3f03ffe7&lt;/Data&gt; 
&lt;/EventData&gt;
&lt;/Event&gt;

The instance log says basically nothing again:

3952: 05:13:14.542 - Failed to create crash dump
3952: 05:13:14.542 - Purecall

When it starts back up it says this:

2476: 05:15:08.733 - Config File: C:\photon\deploy\bin_Win32\PhotonServer.config
2476: 05:15:08.733 - Will NOT produce crash dumps

So apparently I have not set it to create a crash dump as I thought. I have now added the config to ProduceDumps="true" for the instance and will re-start it later to pick that up in the future. No idea why that was not set... but it is now.

Thanks,
Josh

Tobias · January 2012

hey.
Any news on this?
This is sadly one of the issues we can't locate without a dump file, so did the server create some yet?

JoshF47 · January 2012

Tobias wrote:

Any news on this?..

No updates today, it seems to not happen very often as there was about a month gap since the last one. The instance is definitely set to create a crash log now, so if/when it happens again I will let you know for sure - just don't expect it be "soon" (which I suppose is a good thing in one sense and not in another). I completely understand the need for the crash log in order for you to do anything with this.

Thanks for the great support, as always!
Josh

Production Server Crashed - Running Version 2.4.7

Comments