XRGrabInteractable & Pun2 - Gameobject sync issue

Options

I have been trying to use XR Toolkit and Photon 2 Networking to create a multiplayer experience. When Player A grabs an object using XRGrabInteractable it looks fine. Player B, watching that interaction - vice versa.

When Player B grabs the object from the player A hands, for Player B object is in hand and for Player A object is still in the hand (When grabbed from either players hand, object is not syncing in network - both players think they are holding the object)

 public class XRGrabNetworkInteractable : XRGrabInteractable

  {

    private PhotonView photonView;

    private void Start()

    {

      photonView = GetComponent<PhotonView>();

      photonView.OwnershipTransfer = OwnershipOption.Takeover;

    }

    protected override void OnSelectEntered(XRBaseInteractor interactor)

    {

      photonView.RequestOwnership();

      base.OnSelectEntered(interactor);

    }

}