Photon hang when connecting under bad internet connection

KevinB · June 2013

Hi,

I am using Photon iOS SDK v3.0.4.4. I simulate a bad internet connection by unplugging internet connection behind my router. When I do that, photon hangs the main thread when I call ExitGames::Photon::PhotonPeer::connect for like 30sec to 1min. looks like the c library function gethostbyname used by Photon is a synchronize call and when it is called during bad internet connection, it can block for over 30sec.

any suggestion on workaround?

here's the call stack when it hangs:
#0 0x33443604 in kevent ()
#1 0x382fb48a in _mdns_search ()
#2 0x382faa20 in mdns_hostbyname ()
#3 0x382fc586 in search_host_byname ()
#4 0x382f925c in gethostbyname ()
#5 0x00210460 in ExitGames::Photon::PhotonConnect::startConnection(char*) at /Development/egp/photon-sdk-native/branches/v3.0.4.x/Photon-c/src/PhotonConnect.cpp:225
#6 0x00208e9c in ExitGames::Photon::EnetPeer::startConnection(char*) at /Development/egp/photon-sdk-native/branches/v3.0.4.x/Photon-c/src/EnetPeer.cpp:81
#7 0x0020df3e in ExitGames::Photon::PeerBase::connect(wchar_t const*, unsigned char const*) at /Development/egp/photon-sdk-native/branches/v3.0.4.x/Photon-c/src/PeerBase.cpp:174
#8 0x00208e7e in ExitGames::Photon::EnetPeer::connect(wchar_t const*, unsigned char const*) at /Development/egp/photon-sdk-native/branches/v3.0.4.x/Photon-c/src/EnetPeer.cpp:76
#9 0x00207fb6 in PhotonPeer_connect at /Development/egp/photon-sdk-native/branches/v3.0.4.x/Photon-c/src/CPhotonPeer.cpp:88
#10 0x001ff624 in ExitGames::Photon::PhotonPeer::connect(ExitGames::Common::JString const&, unsigned char const*) at /Development/egp/photon-sdk-native/branches/v3.0.4.x/Photon-cpp/src/PhotonPeer.cpp:88
#11 0x00150a94 in ExitGames::LoadBalancing::Client::connect(ExitGames::Common::JString const&) at common/Network/LoadBalancing/src/LoadBalancingClient.cpp:152

Kevin

Kaiserludi · June 2013

Hi KevinB.

I can't reproduce it here. I have tried it with simulator and device, ethernet and wifi, connecting to the public cloud using Photon iOS SDK v3.0.4.4, but gethostbyname() doesn't hang for me and the client immediately tells me that it can't connect.
May be this is only reproducible with some special local network setup.
Could you check please, if you can reproduce it without your router, maybe just by deactivating the network connections directly on the device?

KevinB · June 2013

it can't be reproduced by directly deactivating network connections on the device. our users experience a hang when connecting to a public wifi that is extremely slow.

currently, the only way I can simulate this is by having my iphone connect to my wifi router to the public cloud, and then unplug ethernet cable behind the router. You should check that the iphone still thinks it's connected to wifi. oh also, photon should be connecting to a hostname instead of ip address in order to trigger this. and in this scenario, it would hang.

Let me know if u need more info. Thanks!
K

Tobias · June 2013

The DNS resolution fails and blocks until the standard library throws the error. As it's done on the main thread, everything is blocked.
As workaround you could call Connect() on another thread, I guess. Alternatively DNS resolution can be done in an extra step before calling Connect with the IP.
Kaiserludi will take a look if he can solve this in the library.

KevinB · June 2013

is it ok to call Connect on another thread? just want to make sure coz there is gonna be quite a bit of code change in how we currently use photon since our code is based on the load balancing client example.

Kaiserludi · June 2013

I have researched a bit and it looks like the bsd sockets gethostbyname() function is in general always synchronous and the iOS implementation of it is using a rather huge timeout value that is not adjustable by the caller.
I am currently looking into alternatives to gethostbyname() and would estimate that we could release an SDK with asynchronous DNS resolution later this week.

KevinB wrote:

is it ok to call Connect on another thread?

Currently that's only OK if you are doing all the calls to Photon from that same separate thread. General thread-safety of our native clients is on the todo list, but not yet accomplished.

KevinB · June 2013

Hi guys,

do u guys have ETA on this issue? just wonder if I should wait or work on a workaround on our end. Thanks.

Kevin

Kaiserludi · June 2013

The ETA for an official release is somewhere in the first half of July.

However I could provide you with a prerelease version on Monday, so that you could already check, if it solves your issue. Although the assync approach isn't working on all supported platforms yet, it seems to already work on iOS.

KevinB · July 2013

we tried the prerelease build. looks like photon no longer hangs on resolving the hostname, however under the bad network scenario as stated before, connectReturn callback is never called, as a result it never successfully connect to the server.

wonder if u guys seen this.

KevinB

Kaiserludi · July 2013

I have just sent you another pre-release build via PM. Please try it out and tell us, if it solves that issue.

rogerdpack · November 2019

I'm chasing an issue with a similar trace: https://github.com/crystal-lang/crystal/issues/8376
Anybody know what was done to fix this particular issue? commits or what not? Thanks all!

Kaiserludi · November 2019

Hi @rogerdpack.

We solved this issue by moving the call to gethostbyname() into its own thread, so that it can't block other threads

Photon hang when connecting under bad internet connection

Comments