PUN2 Free: Sync position of player joining a room where other players already moved (teleport)

BFX

I have a VR app where two players A and B can join a common room from a Lobby. The avatars prefab have this configuration:

https://ibb.co/gFFTqnK
https://ibb.co/xFcYBqV

There is a procedure creating LocalAvatar and RemoteAvatar for each player when joining a room. At the beginning, the master client (i.e. A) get positioned in a point X, while the host client (i.e. get positioned in a point Y.

Now let's consider the current scenario

• A connect to the Room (and become Master)
• A LocalAvatar gets created at point X (on A system)
• B connects to the Room (and become guest)
• B LocalAvatar gets created at point Y (on B system)
• B RemoteAvatar gets created at point Y (on A System)
• A RemoteAvatar gets created at point X (on B system)
• A moves around
• B disconnects
• B reconnects
• Now, when B joins the room, A's remoteAvatar gets created in the starting position X instead of the current one
• Unless A teleport somewhere triggering and RPC to update the position there is an off-sync for A's position
• Also, after a while, both players positions seem to go off-sync

Some notes:
1) When creating the RemoteAvatar I'm using the following code:

// Send event to create a remote copy of Local Avatar in all guest clients
            if (PhotonNetwork.AllocateViewID(photonView))
            {
                RaiseEventOptions raiseEventOptions = new RaiseEventOptions
                {
                    CachingOption = EventCaching.AddToRoomCache,
                    Receivers = ReceiverGroup.Others
                };

                SendOptions sendOptions = new SendOptions
                {
                    Reliability = true
                };
                PhotonNetwork.RaiseEvent(InstantiateVrAvatarEventCode, photonView.ViewID, raiseEventOptions, sendOptions);
                Debug.LogFormat("Instantiated Local Avatar [PhotonView:{0}]", photonView.ViewID);
            }

Since I'm using AddToRoomCache, when a player joins a room where there is already another player I should get the latest position of the user. This is not happening

2) As you noticed from the AvatarPrefabs I'm using Reliable as observe option, so I would expect to obtain continuous updates of the avatar position.

3) Objects which get instantiated with the same prefab containing a PhotonView, don't have this problem. If you reconnect to the room you will find them where they were when you left. It looks like the problem is for the avatar since I'm using two different prefabs (LocalAvatar and RemoteAvatar) connected by the same PhotonViewID.
Maybe I need to manually trigger an update of the avatar position/rotation on PhotonTransformView?

4) Both my LocalAvatar and RemoteAvatar Prefab contain a PhotonAvatarView and a PhotonTransformView connected to a PhotonView:

https://ibb.co/gFFTqnK
https://ibb.co/xFcYBqV
https://ibb.co/gZHbsrZ
https://ibb.co/SwjFbDn

PhotonAvatarView.cs (taken from (from https://stackoverflow.com/questions/58137562/pun2-photonavatarview-doesnt-detect-oculus-standard-teleport-event):)

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.IO;
using Photon.Pun;

public class PhotonAvatarView : MonoBehaviour, IPunObservable
{

private PhotonView photonView;
private OvrAvatar ovrAvatar;
private OvrAvatarRemoteDriver remoteDriver;

private List<byte[]> packetData;

public void Start()
{
photonView = GetComponent<PhotonView>();

if (photonView.IsMine)
{
ovrAvatar = GetComponent<OvrAvatar>();
packetData = new List<byte[]>();
ovrAvatar.RecordPackets = true;
ovrAvatar.PacketRecorded += OnLocalAvatarPacketRecorded;
}
else
{
remoteDriver = GetComponent<OvrAvatarRemoteDriver>();
}
}

public void OnDisable()
{
if (photonView.IsMine)
{
ovrAvatar.RecordPackets = false;
ovrAvatar.PacketRecorded -= OnLocalAvatarPacketRecorded;
}
}

private int localSequence;

public void OnLocalAvatarPacketRecorded(object sender, OvrAvatar.PacketEventArgs args)
{
using (MemoryStream outputStream = new MemoryStream())
{
BinaryWriter writer = new BinaryWriter(outputStream);

var size = Oculus.Avatar.CAPI.ovrAvatarPacket_GetSize(args.Packet.ovrNativePacket);
byte[] data = new byte[size];
Oculus.Avatar.CAPI.ovrAvatarPacket_Write(args.Packet.ovrNativePacket, size, data);

writer.Write(localSequence++);
writer.Write(size);
writer.Write(data);

packetData.Add(outputStream.ToArray());
}
}

private void DeserializeAndQueuePacketData(byte[] data)
{
using (MemoryStream inputStream = new MemoryStream(data))
{
BinaryReader reader = new BinaryReader(inputStream);
int remoteSequence = reader.ReadInt32();

int size = reader.ReadInt32();
byte[] sdkData = reader.ReadBytes(size);

System.IntPtr packet = Oculus.Avatar.CAPI.ovrAvatarPacket_Read((System.UInt32)data.Length, sdkData);
remoteDriver.QueuePacket(remoteSequence, new OvrAvatarPacket { ovrNativePacket = packet });
}
}

public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info)
{
if (stream.IsWriting)
{
if (packetData.Count == 0)
{
return;
}

stream.SendNext(packetData.Count);

foreach (byte[] b in packetData)
{
stream.SendNext(b);
}

packetData.Clear();
}

if (stream.IsReading)
{
int num = (int)stream.ReceiveNext();

for (int counter = 0; counter < num; ++counter)
{
byte[] data = (byte[])stream.ReceiveNext();

DeserializeAndQueuePacketData(data);
}
}
}
}

This system works well, but the standard oculus teleport doesn't get detected and I was forced to write a separate script to keep Local and Remote avatar position updated when there is a teleport. My theory is that the teleport performed by Oculus works on a higher level in the hierarchy and doesn't get synchronized by PhotonAvatarView. That's why if I leave the room and re-enter the remote and local avatars positions go off-sync


History
Looks like somebody else had this problem in the past: Please check this, this and this

Also, could this be a solution? https://stackoverflow.com/questions/42312692/how-to-get-the-position-of-all-the-player-in-photon

Thanks!