Client input not updated on server

Hello,

I have a 2-players multiplayer VR game where the input of the controllers (pos/rotation) dictates the input of the player, and that's what I'm pooling in the OnInput method.

For the 1st player (host) it works perfectly fine, and when I enter as a spectator to the scene, I can see the movement of the host properly.

However, when I join the game as a client (using Auto host or client mode), as far as I was able to analyzed the case -

The host sees no input, so the objects that the should follow the controllers are not reacting to the movement of the client, meaning the client does not send input.

And, on the client side, I am able to see the host properly, but the client's controllers are extremely jittery (I assume it's due to the host "fixing" the client prediction that is not matching what its expecting.)

I have also checked and saw that the controllers input data that I am sending (input.Set) are indeed the correct position.

I have no more things to try, I have spent nearly two weeks day to night trying to figure this (appearently) small issue, with no success.

Any help would be greatly appreciated, the relevant code for collecting the input attached below.

InetworkInput struct:

public struct NetworkInputData : INetworkInput

{

  public Vector3 controllerLPos;

  public Quaternion controllerLRot;

  public Vector3 controllerRPos;

  public Quaternion controllerRRot;

}

Setting Input:

  public void OnInput(NetworkRunner runner, NetworkInput input)

  {

    if (!networkObject.HasInputAuthority)

      return;

      var myInput = new NetworkInputData();

      myInput.controllerLPos = gloveL.FollowTransform.position;

      myInput.controllerLRot = gloveL.FollowTransform.rotation;

      myInput.controllerRPos = gloveR.FollowTransform.position;

      myInput.controllerRRot = gloveR.FollowTransform.rotation;

      input.Set(myInput);

  }

Implementing Movement:

  public override void FixedUpdateNetwork()

  {

    if (networkObject.HasInputAuthority)

    {

      if (GetInput(out NetworkInputData data))

      {

          controllerPos = data.controllerRPos;

          controllerRot = data.controllerRRot;

        }

   }

      base.PhysicsMove();

    }

  }