Server freeze - channel timeout

Options
2»

Answers

  • BesoC
    BesoC ✭✭
    Options
    @chvetsov Thanks, I completely missed that topic and will give it a try.


    Meanwhile, can you please advise about using async/await and Task.Run(async () => await myAsync()).Wait(); patterns inside a PoolFiber? Is it safe or will it add instabilities by, say, messing somehow with the way Photon manages threads?
  • chvetsov
    chvetsov mod
    Options
    We do not manage thread in any special way. We just use ThreadPool.EnqueAction or something like this.
    But from what I see you still block. so usage async stuff does not make any difference in this case.

    if you will just use await, then you should be aware that part of the method before await will be executed on fiber, the part after NOT. this may break thread safety. the part after await can be placed into fiber using lambda. In this case, it will work in thread-safe mode. But I'm not sure about your logic

    best,
    ilya
  • BesoC
    BesoC ✭✭
    edited April 2019
    Options
    Please see next comment
  • BesoC
    BesoC ✭✭
    edited April 2019
    Options
    @chvetsov you mean, something like this would be thread-safe? 

    
fiber.Enqueue(() =>
{
    Task.Run(async ()=> await myAsync()).Wait(); 
    fiber.Enqueue(()=> 
    {
         DoSomethingSync1();
         DoSomethingSync2();
     });
}
  • BesoC
    BesoC ✭✭
    edited April 2019
    Options
    @chvetsov Following instructions on the page you wrote Photon Server Stack Overflow


    When debugging dump in VS2017, process hits the breakpoint, but cannot load symbols from Photon.SocketServer.pdb
  • chvetsov
    chvetsov mod
    Options
    > When debugging dump in VS2017, process hits the breakpoint, but cannot load symbols from Photon.SocketServer.pdb
    You do not need them. The issue is in your code

    I think your code should look like this
    For instance, you may have code in Peer.OnOperationRequest

    ......
    await myAsync();
    RequestFiber.Enqueue(()=>
    {
    DoSomethingSync1();
    DoSomethingSync2();
    });
    .................

    You should be aware that after await myAsync(); peers fiber will execute the next action from his list if any. Then when that line is finished, a new action will be enqueued to RequestFiber. You need this only in case if DoSomethingSync1 and 2 access some peers data

    best,
    ilya
  • BesoC
    BesoC ✭✭
    edited April 2019
    Options
    @chvetsov Understood. We're gonna make a few stress tests considering your advises. Thanks for your responsiveness :)

    Meanwhile, can you please take a look at this log? 
    
7316: 00:14:36.085 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000BB03190
13904: 00:43:04.148 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000BB04E70
12460: 00:50:31.874 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000BB144D0
13904: 11:28:22.247 - CTCPWebSocketServer::OnError() - CFilteringStreamSocketConnectionManager::FilterReadCompleted() - Exception: CFilteringStreamSocketConnectionManagerBase::RequestWrite() - Socket is closed: 0000000010342650
13904: 11:28:22.247 - CFilterDataBase::OnSocketReleased() - References still held: 2
13904: 12:24:58.838 - CTCPWebSocketServer::OnError() - CFilteringStreamSocketConnectionManager::FilterReadCompleted() - Exception: CFilteringStreamSocketConnectionManagerBase::RequestWrite() - Socket is closed: 0000000010926FF0
13904: 12:24:58.838 - CFilterDataBase::OnSocketReleased() - References still held: 2
13904: 12:55:37.648 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000C004650
13904: 13:02:13.639 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000C002970
13904: 13:02:20.780 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000BA54240
7316: 13:38:00.943 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000035D4EAE0
7316: 13:39:04.953 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000035D59AC0
10184: 13:41:17.107 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 00000000246DD710
10184: 13:41:57.887 - CTCPWebSocketServer::OnError() - CFilteringStreamSocketConnectionManager::FilterReadCompleted() - Exception: CFilteringStreamSocketConnectionManagerBase::RequestWrite() - Socket is closed: 00000000361BA3E0
10184: 13:41:57.887 - CFilterDataBase::OnSocketReleased() - References still held: 2
12460: 13:47:36.629 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000023E5C110
12460: 13:51:34.270 - CTCPWebSocketServer::OnWriteCompletionError() - Flow control - Connection aborted due to - An existing connection was forcibly closed by the remote host.
12384: 14:10:47.987 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000035C7E170
14180: 14:16:42.178 - CTCPWebSocketServer::OnWriteCompletionError() - Flow control - Connection aborted due to - An existing connection was forcibly closed by the remote host.
12384: 14:21:59.358 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000003609C4B0
14180: 14:37:23.193 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000002449FCA0
12384: 14:59:28.206 - CTCPWebSocketServer::OnError() - CFilteringStreamSocketConnectionManager::FilterReadCompleted() - Exception: CFilteringStreamSocketConnectionManagerBase::RequestWrite() - Socket is closed: 000000000BB166F0
12384: 14:59:28.206 - CFilterDataBase::OnSocketReleased() - References still held: 2
12384: 15:17:37.910 - CTCPWebSocketServer::OnError() - CFilteringStreamSocketConnectionManager::FilterReadCompleted() - Exception: CFilteringStreamSocketConnectionManagerBase::RequestWrite() - Socket is closed: 0000000035D4FAA0
12384: 15:17:37.910 - CFilterDataBase::OnSocketReleased() - References still held: 2
14180: 16:07:10.324 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000035D52C80
12384: 16:23:20.398 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000010925D90
14180: 16:28:05.548 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 0000000035745A00
12384: 16:36:59.242 - CTCPWebSocketServer::ReadCompleted - Exception - TWebSocketBase<BaseInterface>::Read() - Socket is closed: 000000000BB13CF0
    AFAIK, these lines indicates some socket-related exceptions - read attempt from closed socket, maybe leaks, etc. Can you please help us in with this as well?
  • chvetsov
    chvetsov mod
    Options
    @BesoC those exceptions reflect normal flow. Because sockets can disconnect at any moment of time. Nothing wrong is with your code or with our

    best,
    ilya
  • BesoC
    BesoC ✭✭
    edited May 2019
    Options
    Hi @Ilya
    We have updated our server according to your advises, avoided blocking calls and removed channels at all. Everything works very well. Thanks a lot for your help :)
  • chvetsov
    chvetsov mod
    Options
    cool, thank you

    best,
    ilya

Cookies,
anyone?

We use cookies and related technologies to enhance your experience, show you personalised content and analyze performance and traffic on our website. By clicking on the „Accept All“ button you consent to the use of non-functional cookies and the subsequent processing of personal data to optimize our website and services as described in more detail in our Privacy and Cookie Policy.

By clicking on the „Customize or Deny all“ button you can decide otherwise. Clicking on the „Customize“ button will take you to a page where you can configure the usage of non-functional cookies (and related technologies) or deny all of them. You can access these settings at any time and also subsequently deselect cookies at any time in the footer area of our website.

ACCEPT ALL

Cookie Overview

We use the following categories of cookies and related technologies to enhance your experience, show you personalized content, and analyze performance and traffic on our website. We respect your right to privacy and accordingly you can chose to not allow some types of cookies (and related technologies). Click on the different category sliders and change our default settings to manage your cookie settings.

For more information on the specific cookies/related technologies we use and on how we use these, please see section 15 E. of our Privacy and Cookie Policy.

ABSOLUTELY NECESSARY

Authentication cookies we use are required to run our services … Cookies are required: Link

FUNCTIONAL AND MARKETING COOKIES

These cookies collect anonymous data and allow us to optimize our website and user experience. These cookies are listed here:

ANALYTICS

Help us to understand how visitors interact with our services, enables us to analyze and improve our services (also through third party analytics).