AppDomain.CurrentDomain.UnhandledException is Not Called
Options
Shakaron
✭
I've got a similar problem as described in thread https://forum.photonengine.com/discussion/9955/unhandled-exception-handler-not-called.
The difference is that we would like to do more than just logging: we would like to create a memory dump so any crash could be analyzed.
The server application now closes if an exception happens (opposed to the "Ignore" option), but the UnhadnledException callback is still not called.
The version of the Photon Server I'm running is 4.0.29.11263
Any help would be appreciated.
The difference is that we would like to do more than just logging: we would like to create a memory dump so any crash could be analyzed.
public class MyApplication : ApplicationBaseI made sure the the Unhandled Exception Policy is configured as "TerminateProcess" so Photon will not swallow all the exceptions. This information was available here: https://doc.photonengine.com/en-us/server/current/app-framework/exception-handling#configuring_the_unhandledexception_policy
{
protected override void Setup()
{
// ...
AppDomain.CurrentDomain.UnhandledException += OnUnhandledException;
// ...
}
private static void AppDomain_OnUnhandledException(object sender, UnhandledExceptionEventArgs args)
{
_log.Fatal(args.ExceptionObject);
MemoryDump.MiniDump.Create(Process.GetCurrentProcess(), Path.Combine(Path.GetTempPath(), "MyApplication.dmp"));
}
}
The server application now closes if an exception happens (opposed to the "Ignore" option), but the UnhadnledException callback is still not called.
The version of the Photon Server I'm running is 4.0.29.11263
Any help would be appreciated.
0
Comments
-
Hi Ilya,
thanks for the quick response.
I've added a simplethrow new Exception("ExceptionForTesting");
into the above mentioned Setup() as the last line.
The application crashes as expected, as the exception handling policy is still "TerminateProcess". However, I didn't see any crash dump file created in Photon-OnPremise-Server-SDK_v4-0-29-11263\deploy\bin_Win64\log\ folder. Only two log files there: PthotonCLR.log and Photon-MyApplication-20190906.log
The latter contains lines, such as
"29468: 20:28:43.051 - Will produce at most: 10 crash dumps"
but apart from that, I didn't see any trace of memory dumps. And after this line, there was no info, where the dump is created, if any.
29468: 20:28:52.039 - PulseIQ:2 - PhotonRunning() failed.
Exception:
System.Exception: ExceptionForTesting
at MyApplication.Setup() in C:\_______\MyApplication.cs:line 196
at Photon.SocketServer.ApplicationBase.PhotonHostRuntimeInterfaces.IPhotonControl.OnPhotonRunning() in h:\svncontent\photon-socketserver-sdk_cloud\src\Photon.SocketServer\ApplicationBase.cs:line 1189
at PhotonHostRuntime.PhotonDomainManager.PhotonPlainAppDomainBehavior.PhotonRunning()
at PhotonHostRuntime.PhotonDomainManager.PhotonRunning()
29468: 20:28:52.040 - CService::OnException() - Exception: CManagedHost::PhotonRunning() - Failed in AppDomain: 2 - ExceptionForTesting0 -
Hi Ilya,
I think I was in a bit of a rush on Friday when I posted my original post. Let me share my update on the subject.Update:
I've done a bit more testing, at it seems like no matter what I set for UnhandledExceptionPolicy (ReloadAppDomain, Ignore, or TerminateProcess), the application always closes and callsPhoton.SocketServer.ApplicationBase.TearDown()
.
This only happens if I throw an exception inPhoton.SocketServer.ApplicationBase.Setup()
.
If I try to throw an exception inPhoton.SocketServer.Rpc.Peer.OnOperationRequest()
, the exception is always swallowed, regardless what the UnhandledExceptionPolicy setting is.
I've also found settings how to set up the Memory Dump: https://doc.photonengine.com/en-us/server/current/app-framework/exception-handling#photon_core_debugging
Unfortunately, this doesn't work either. I've tried it with the above variations: all 3 UnhandledExceptionPolicy policy, throwing exception inPhoton.SocketServer.ApplicationBase.Setup()
andPhoton.SocketServer.Rpc.Peer.OnOperationRequest()
. Never saw any memory dump in the Photon-OnPremise-Server-SDK_v4-0-29-11263\deploy\bin_Win64\log\ folder.Note #1:
The SDK says version 4.0.29.11263, but if I runPhotonSocketServer.exe /?
it saysPhoton Socket Server - 4.0.28.2962
So it's a slightly different version that what the SDK indicates.
Usage: PhotonSocketServer.exe Followed by the following command line options.
/help Display this list.
/noMessages Must be first. Do not display status messages.
/install name Installs an instance of the service named 'name'.
/installCounters Installs performance counters for this service.
/remove name Removes the instance of the service named 'name'.
/removeCounters Removes performance counters for this service.
/run name Runs the service as a normal process for debugging.
/stop Request all instances of Photon to shutdown and exit.
/stop1 name Request instance 'name' of Photon shutdown and exit.
/configPath path Use the supplied path to locate the config file.
/config file Use the supplied config file.
Note that /config and /configFile can be used with /install and /run
Note that only /noMessages can be combined with all other command line options.Note #2:
I'm pretty sure my configuration is all right, as I've noticed if I mess up something in the configuration file (e.g. wrong value), Photon doesn't start.
So the configs seem to be correct(as Photon starts), but neither the MemoryDump nor the UnhandledException settings work, nor does AppDomain catch any unhandled exception (as Photon handles them anyway), so I'm running out of options. Anything else I could try? Any idea why these settings make no difference?
Is there a sample project that shows that these settings work as intended?0 -
hi, @Shakaron
There is one feature related to fibers. By default, peers are using fiber with FailSafeExecutor. So, if some exception happens during fiber's action execution, it is, yes, swallowed to not break room/fiber workflow, but it is also logged. Almost everything is going on in fibers, so that is the reason why you do not see unhandled exceptions. They all are handled.
This is how we work with all that stuff.
if you want to change this, you have to override ApplicationBase.CreatePeerFiber. By default, it creates PoolFiber with FileSafeExecutor. You may use DefaultExecutor or your own executor.
best,
ilya0 -
Hi, Ilya.
I've trying to use theDefaultExecutor
to customize the IFiber creation. Still not exception caught/logged coming forPhoton.SocketServer.Rpc.Peer.OnOperationRequest()
.
Then I created my own IExecutor:public class MyApplication : ApplicationBase
I can see a new
{
private class MyExecutor : IExecutor
{
public void Execute(List toExecute)
{
foreach (var action in toExecute)
Execute(action);
}
public void Execute(Action toExecute)
{
try
{
toExecute();
}
catch (Exception e)
{
log.Fatal($"Exception in ThreadFiber: {e}");
throw;
}
}
}
protected override IFiber CreatePeerFiber(InitRequest request) => new PoolFiber(new MyExecutor());
}PoolFiber
withMyExecutor
being created. I can seetoExecute()
being called.
However, I never catch the exception, never get to the linelog.Fatal(...)
A quick glance at the call stack can probably explain why:> MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.CompWorld.OperationHandlerWorld.OnHandlerMessage(MyApplication.Local.Common.SubCode subcode, Photon.SocketServer.OperationRequest request, Photon.SocketServer.OperationResponse response, MyApplication.Local.ControlServer.CompClientPeer.BaseClientPeer peer, Photon.SocketServer.SendParameters sendParameters) Line 26 C#
Please, notice the [External Code] between
MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.CompClientPeer.BaseClientPeer.OnOperationRequest(Photon.SocketServer.OperationRequest operationRequest, Photon.SocketServer.SendParameters sendParameters) Line 61 C#
[External Code]
MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.ControlServerApplication.MyExecutor.Execute(System.Action toExecute) Line 196 C#
MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.ControlServerApplication.MyExecutor.Execute(System.Collections.Generic.List toExecute) Line 188 C#
[External Code]OnOperationRequest
andMyExecutor.Execute
.
That external code (probablyPeer
) can easily handle the exception, preventing it to leave the execution, preventing it to be caught on the level ofMyExecutor
.
Please, find the logs here0 -
Hi Iilya,
I've tried to use theDefaultExecutor
, but nothing changed. I even created my own executor:public class MyApplication : ApplicationBase
I can see the
{
private class MyExecutor : IExecutor
{
public void Execute(List toExecute)
{
foreach (var action in toExecute)
Execute(action);
}
public void Execute(Action toExecute)
{
try
{
toExecute();
}
catch (Exception e)
{
log.Fatal($"Exception in ThreadFiber: {e}");
throw;
}
}
}
protected override IFiber CreatePeerFiber(InitRequest request) => new PoolFiber(new MyExecutor());
}CreatePeerFiber
being called.Execute
being called.toExecute()
being called. Yet I never get into the catch branch, when I throw an exception inPhoton.SocketServer.Rpc.Peer.OnOperationRequest
.
Looking at the call stack may reveal the culprit here:> MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.CompWorld.OperationHandlerWorld.OnHandlerMessage(MyApplication.Local.Common.SubCode subcode, Photon.SocketServer.OperationRequest request, Photon.SocketServer.OperationResponse response, MyApplication.Local.ControlServer.CompClientPeer.BaseClientPeer peer, Photon.SocketServer.SendParameters sendParameters) Line 26 C#
As you can notice there is an [External Code] call between
MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.CompClientPeer.BaseClientPeer.OnOperationRequest(Photon.SocketServer.OperationRequest operationRequest, Photon.SocketServer.SendParameters sendParameters) Line 61 C#
[External Code]
MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.ControlServerApplication.MyExecutor.Execute(System.Action toExecute) Line 196 C#
MyApplication.Local.ControlServer.dll!MyApplication.Local.ControlServer.ControlServerApplication.MyExecutor.Execute(System.Collections.Generic.List toExecute) Line 188 C#
[External Code]OnOperationRequest
andMyExecutor.Execute
. That must bePeer
invokingOnOperationRequest
and handling the exception. So it seems like I will never be able to have a convenient top level class to catch and handle the exceptions.
It seems like I have to push down the exception handling, logging and dump file creation to every individual call when I may have an exception.
If it is of any help, please, find the logs here
No exception logged there. No dump file created (Dump and Exception settings are same as before).0 -
hi, @Shakaron
log you have provided is native log. it does not contain anything about managed exception. It can be useful if you have stackoverflow
please check PhotonCLR.log in same folder or 'your_root_folder/log'
your exceptions are caught and logged here
```
protected void OnOperationRequestInternal(OperationRequest operationRequest, SendParameters sendParameters)
{
try
{
this.OnOperationRequest(operationRequest, sendParameters);
}
catch (Exception e)
{
log.Error("Exception during handling of operation", e);
}
}
```
so, make sure that logging is correctly setup in your application.
best,
ilya0 -
Hi, @Shakaron
That is totally ok because we intercept exception earlier. See the example I gave in the previous post.
PhotonCLR will contain only unhandled exceptions. You have to setup logging for your application to be able to see logged errors. Just copy log setup from LoadBalancing applications code.
best,
ilya0