Oculus Quest 2 with hand tracking, grabbable objects not synced.

Matias
2020-11-24 14:45:39

Hi guys, I'm working in VR with the Oculus Quest 2 using Unity 2019.4.14f1 LTS version + PUN 2.

I managed to make hand tracking work and send the hands position and rotation through the network by using this plugin and the help of the developer. The hand's movement works really well without any lag or major latency problems.

Now... I'm having problems related to the synchronization of grabbable objects.

Let's say player1 and player2 are sharing a room. In front of them, there's a cube. If player1 grabs the cube and moves It with his hand, he will see the cube moving around without any problem, but for player2 the cube will instantly disappear as soon as player1 pinch the cube and move it. The cube will be visible for player2 again, only if player1 releases the object, which will suddenly appear in the position where player1 stopped grabbing the cube.

Everything else is working really well, the hand tracking, the avatars, the objects, voice, chat, etc.
I'm using modifications of the OVR grabber and OVR grabbable Oculus SDK scripts.

This is how the components set up for the grabbable object (a cube).
https://ibb.co/drJtF8P

And this is the components set up for the hand prefab I'm using.
https://ibb.co/SPS16wJ

Do you have any clue about this?

Comments

Matias
2020-11-24 14:47:48

This is my custom PunOVRGrabber.cs script.

using Photon.Pun;
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

namespace Networking.Pun2
{
[RequireComponent (typeof (PhotonView))]
public class PunOVRGrabber : OVRGrabber
{
PhotonView pv;

}

Matias
2020-11-24 14:48:50

And this is my custom OvrGrabbable.cs script:

using UnityEngine;
using UnityEngine.Events;
using Photon.Pun;

//
//Custom grabbable script which checks if the grabber "is mine" to actually grab
//

namespace Networking.Pun2
{
[RequireComponent(typeof(PhotonView))]
public class PunOVRGrabbable : OVRGrabbable
{
public UnityEvent onGrab;
public UnityEvent onRelease;
[SerializeField] bool hideHandOnGrab;
PhotonView pv;
Rigidbody rb;

}

Matias
2020-11-25 12:57:59

Anyone?

Thomukas1
2020-12-01 12:53:44

I think it's really important to know that happens with your events "ongrab" and "onrelease". I am really intrigued that you are tackling this problem. Please consider joining my discord server https://discord.com/invite/G7eE9C9RRg I believe I can help you with this, if you give me more info.

emotitron
2020-12-02 02:35:45

There are some undocumented and not even complete implementations of oculus support in the Simple library (just got added to Pun2) in the SyncOculus folder you can poke around at.

You need to add the define OCULUS to the unity project though, and unpack the Oculus scene that is in the SyncOculus folder to get to it.

You can join our Discord to get some help messing with what I have going in that scene.
https://discord.gg/egaRfd8