Prospects of using Photon Master Server as a NAT relay server

mantracker
edited January 2018 in Native
Hi, our UE4 game currently is running multiplayer using P2P, and we have recently integrated Photon into our game. Currently, our biggest problem so far is handling NAT punch problems. Because Unreal doesn't offer any solutions in terms of handling NAT punch problems, we are considering using Photon as a relay server to do this.

Now, I understand that if we switch our Unreal Netcode to using Photon (rewriting all our netcode, replication and stuff), we will use Photon's client server architecture and will no longer have problems with NAT. However, this is infeasible for our game and I am thinking of another possible solution, which I will discuss below:

From reading about how to NAT punch from here, I am wondering if I can use Photon as a Master server, and let game client and server connect to it to do NAT punch.

Game Client (Behind Router) Public IP:PORT => 1.10.101.116:7777
Game Server (Behind Router) Public IP:PORT => 1.10.100.54:7777
Photon Master Server (Not behind Router)

Prerequistes (Game Client and Server are in the same Photon lobby, already connected together)

1. Game Client finds Game Server in Server list (Sees 1.10.101.116:7777 in server list)
2. Game Client calls opRaiseEvent (Wants to connect to 1.10.101.116:7777)
3. Photon Master server relays this message to 1.10.101.116:7777
4. Game Server receives message, calls opRaiseEvent (First message to 1.10.100.54:7777)
5. Game Client receives message, sends message back to Game Server, connection now works

As far as I understand, this seems to be how NAT punch is supposed to work, and how I can use Photon Server as a relay to do this.

Does this seem feasible? Is this a good idea? Right now in LoadBalancingListener.cpp, it seems that we can't specify which port we want to connect to the Photon Master Server with. Where can we specify exactly which port on client side we want to connect with?

Comments

  • Hi @mantracker.


    Where can we specify exactly which port on client side we want to connect with?

    Sorry, but I am afraid that the client side port is rather random, NOT configurable and also not exposed in the Client API.

    However it does not matter which port the client is using to connect to Photon. You can (and should) just use a different port for the direct connection than for the Photon connection.

    So it should look the following:
    1. Game Sever creates a Photon room and sets a room property with key "hostIP" and value "1.10.101.116:7777".
    2. Game Client finds the room in the room list (Sees a room with room property 1.10.101.116:7777 in the list).
    3. Game Client join that Photon room.
    4. Game Client calls opRaiseEvent() with 'targetplayernr' set to the Game Servers player number and provides its IP address and port that it intends to use for the direct connection as payload of the event.
    5. Photon relays this message to gameServer:photonClientPort.
    6. Game Server receives the Photon event and reads out the IP and port that Game Client is intending to use (for example 1.10.100.54:6666).
    7. Game Client sends something directly (peer2peer without Photon as relay in between) from 1.10.100.54:6666 to 1.10.101.116:7777.
    8. Game Server sends something directly (peer2peer without Photon as relay in between) from 1.10.101.116:7777 to 1.10.100.54:6666.
    9. The direction connection now works.

    Gotcha:
    Steps 7 and 8 need to happen in parallel. One side should not wait for the others message to arrive before sending its own message. The reason for this is that the router won't let the remote clients message through until the local client has sent out a message through that port first. For the same reason the first message of one of the clients might not go through at all (if client As message arrives at the router of client B, before client B has sent out its own direct message, then the router would not let client As message through).
    However all further messages after both clients have sent a first message will go through as long as the routers see the connection as still alive and don't clean it up (so make sure that each side sends keep-alive ping messages regularly whenever it doesn't have to send any regular message for a while (the maximum intervals in which you need to send a message to keep the connection alive would depend on the router models, but sending at least one message per second should be more than enough for all routers)).
  • mantracker
    edited January 2018
    Hi @Kaiserludi

    Thanks for your detailed response! This is less of a question related to Photon but, how would I ensure that step 7 and 8 happen in parallel if they can't communicate to each other yet? Is there a way for Photon to time their relay from step 5 (Say 5 seconds or so) so that the Game Client can wait exactly 4 seconds after sending opRaiseEvent() to try to connect to the Game Server?
    Or is there a better way?

    Secondly, when my Game Client connects to the Photon Master Server, the Photon Master Server needs to store both the public and private IP that the Game Client connected with. For example, if the Game Client is behind a router and its private IP/Port that it uses to connect to Photon is 1.10.100.116:7777, I need Photon Server to store both the private IP/port (1.10.100.116:7777), and the public IP/port that is used by the router (say 54.100.55.55:8888). Then, I need this private and public IP/port pair to be sent back to the Game Server. Is this something doable using Photon?
  • Hi @mantracker.

    When I wrote "parallel", I didn't mean "at exactly the same point in time", but rather "one side should not be waiting to receive a message from the other side before sending one itself". It does not matter if step 7 or step 8 happens first.

    Just send the both IPs to the Photon server. From Photons perspective the IP addresses that sou set as room properties or send by opRaiseEvent() are really just strings. You can just send a string-array of 2 elements with opRaiseEvent() or a Hashtable or Dictionary with 2 entries would also work. For room properties their values can also be of any of the types that are supported by Photon.
  • mantracker
    edited January 2018
    Hi @Kaiserludi

    The problem is the game client won't know what the public port the client used to connect with Photon. The game client knows about the private/public IP, and also the private port, but the router might change the public port used tho.

    Thats why I am wondering whether Photon can detect which public IP and port was used to connect to it, and send it back via some client API call or something.

    In the example I gave, the game client can send 1.10.100.116:7777 and the public IP 54.100.55.55 over to Photon, but the port 8888 is something the game client doesn't know about.
  • Hi @mantracker.


    The game client knows about the private/public IP, and also the private port, but the router might change the public port used tho.

    Thats why I am wondering whether Photon can detect which public IP and port was used to connect to it, and send it back via some client API call or something.

    No, this is currently not accessible through Photon.
  • PS:
    What is your schedule and budget?

    We are currently internally working on a new product flavor that adds some additional features, including using Photon as matchmaking and relay server for games that use Unreal networking. However this is still in early development and it will take a couple more months until it becomes available. Furthermore that product flavor will be targeting big studios, not small indie developers and its pricing will reflect this. This solution won't make sense for you if you are not capable or willing to pay a couple thousand dollars.
  • PS:
    What is your schedule and budget?

    We are currently internally working on a new product flavor that adds some additional features, including using Photon as matchmaking and relay server for games that use Unreal networking. However this is still in early development and it will take a couple more months until it becomes available. Furthermore that product flavor will be targeting big studios, not small indie developers and its pricing will reflect this. This solution won't make sense for you if you are not capable or willing to pay a couple thousand dollars.

    Unfortunately our team needs this solution in something like 2 months. So we probably won't be able use Photon for this solution. Eitherway, thanks for your help. I'll see if I can find other solutions for this.
  • Hello @mantracker

    You are absolutely right about the punch process. Currently, the connection that Photon client make with the Photon server it is meant only for sending and receiving relayed messages.

    There is no direct way or exposed method that shows the public IP of the Photon client, it can not know this info looking at the local socket. Even if you are working directly with sockets, there is no easy way to get this data.

    You can take a look at how the WebRTC architecture works, as it makes direct connections between clients also using a relay server.
  • hi, is it still not possible to use Unreal networking code while using Photon for matchmaking, relay server and NAT traversal
  • Hi @MrAndrey.

    UE4 allows you to write your own online subsystem:
    https://docs.unrealengine.com/en-US/Programming/Online/index.html

    You can write an Online Subsystem that uses Photon and you can then instruct the UE4 Networking APIs to transfer all data to that subsystem.

    I know of developers who have successfully implemented that, but it involves some work to do this.