Has anyone tried to serialize Oculus Avatar?

Hi all, I'm just ramping up on both Oculus Avatar and Photon. I've been reading the getting started tutorial. Is it possible to use the PhotonView scripts to automatically serialize an Oculus avatar?

Comments

  • Hi @RickBlacker,

    which 'getting started tutorial' did you read? If you haven't done this already, I recommend you reading this guide to see how you can set up the Oculus Avatar SDK with Photon.
  • I am trying to get this to work as well, and I followed that guide you posted and it doesn't work at all. Is there a guide that actually walks you through how to get it working because that current guide leaves alot to be desired.
  • Hi @SirisDracken,

    please describe what kind of problems you have with this guide so we can update and fix the provided information. Our tests were working very well.
  • Hello @Christian_Simon,

    Sorry for the late reply, was out on vacation for Thanksgiving. :)

    I have a pretty simple project setup at the moment, that only has the Oculus Platform, OVR, OvrAvatar, Photon Unity Networking, and Photon Chat Api integrated into the project.

    In addition to that I have a simple "NetworkManager" script that is built off of the "ConnectAndJoinRandom" script supplied in the Photon Unity Networking Utility Scripts, that I have modified to just take the stuff you were doing in an update loop and put that in the Start() to get it to join a random room immediately on play.

    For testing purposes, I also added a UnityEvent to the OnJoinedRoom() which I have connected to an Avatar Spawner Script I have attached to the same GameObject, to make it spawn the OVRCameraRig (Which has my "AvatarManager" script on it), or my Test Camera (if it is a test build, so I can test on one machine) once the room has actually been joined.

    I followed your guide and after the first section I was having difficulty following what additional scripts I was supposed to be creating, but I ended up making an "AvatarView" script that is attached to both my Local Avatar, and my Remote Avatar. This script has everything from your guide up to the "Instantiating Avatars" section, including having a Photon View on both of my prefabs that has the Observed Components set to the Local Avatar script.

    From this point in the guide onwards its very unclear what needs to be done and where it needs to be done. First the guide does not specify where the Avatar Spawning should be done. Is is a new seperate script? Should it be part of my "NetworkManager"? I didn't know so I guesssed and made a seperate "AvatarManager" script. I did exactly what your guide says to do, and...it didnt work.

    My first issue was that your guide does not mention subscribing to the OnEvent() call. As someone new to your system, and someone following a guide, that tripped me up in trying to get this to work for the first time, if you could please add this part to the guide that would be nice for us newer folks and would mean that by following the guide it would tell me everything I needed to do to make it work.

    This was all that needs to be added to the guide to make it clear:

    public void Start()
    {
    PhotonNetwork.OnEventCall += this.OnEvent;
    }

    I then fixed that and subscribed to the event, and saw that it was now being properly called, yay!

    Ok with that call actually working now, I then I made two seperate scenes, one that is set to spawn my test camera that looks towards the origin so I can see if the remote avatar is working or not. This scene by itself works great, room is created, test avatar is spawned. My other scene is the normal scene, that spawns the OVRCameraRig and the Local/Remote Avatars. By itself this main scene works great, it creates a room if it can't find one, and spawns the OVRCameraRig and Local Avatar which all track properly.

    Then if I make two separate builds one with my main scene, and one with the test scene, I run the test scene project first and it creates a room. I run the main scene, it will spawn the OVRCameraRig, and the Local Avatar just fine, however it will not spawn my Remote Avatar.

    To make sure the two game instances are actually connecting I made a simple cube spawning script that spawns cubes over the network, and sure enough both users are in the same session and can spawn cubes. For the Oculus build, it does indeed spawn the camera rig properly, and it does spawn the Local Avatar but it does not spawn the remote avatar.

    I suspect its because in the guide it never tells me where to put the avatar spawning script so I am unsure if I did that part correctly or not, I put mine on the OVRCameraRig. I also suspect that it doesn't work because your guide says to manually spawn the Local and Remote Avatars, but the Local spawning would only ever happen on my Oculus build, and so the Test build would never run that code because it doesnt have that script to run, which your guide says it wont need.

    I had thought my issue might be that I was spawning the Camera Rig, or Test Camera after the networking script had joined a room. I then took that part out and placed the OVRCameraRig, and the test camera into their scenes directly and I got the exact same result.

  • SirisDracken
    edited November 2017
    Oh one other thing @Christian_Simon,

    The guide has the reader put the Avatar Driver scripting for the AvatarView into the Awake(), this does not work and as other forum posts have noted it needs to instead be put in the Start().

    Until you do this it just spits out errors every frame when you hit play.
  • Ah ha, yup my suspicion was correct, well one of them, lol. So my issue was that my test scene was setup like the guide says, in that it was just a basic scene with a camera, so obviously the spawn code wasn't going to be fired.

    The guide should be updated to mention that for a simple testing project, your simple testing camera or some other object in the scene will need the script to spawn the remote avatar.
  • Hi @SirisDracken,

    thank you for the input.

    First the guide does not specify where the Avatar Spawning should be done.


    You are right about that. I have updated this to make it more clearly that this is another script and doesn't belong to the PhotonAvatarView.

    My first issue was that your guide does not mention subscribing to the OnEvent() call.


    To avoid further misunderstandings I have added a reference to the RaiseEvent documentation page and also code examples showing how to deal with the OnEvent callback.

    I suspect its because in the guide it never tells me where to put the avatar spawning script so I am unsure if I did that part correctly or not, I put mine on the OVRCameraRig.


    Do you mean the OnEvent callback when talking about the 'avatar spawning script'? If so, this can be placed mostly everywhere, for example on a Network Manager you have mentioned, too.

    The guide has the reader put the Avatar Driver scripting for the AvatarView into the Awake(), this does not work and as other forum posts have noted it needs to instead be put in the Start().


    I'm sure that I got this working before creating the guide, strange. However I have added the alternative option to work around the problem, too.

    So my issue was that my test scene was setup like the guide says, in that it was just a basic scene with a camera, so obviously the spawn code wasn't going to be fired.


    You are right, this is a little bit confusing. For the remote client the described setup is basically correct. You only have to add the OnEvent handler in order to instantiate the RemoteAvatar. I'm taking a look at that and see if I can improve that.

    Thanks again for your input.
  • @Christian_Simon

    Yeah no problem, thanks for the quick responses! Glad I could help, and I hope it makes it easier for new people to get it all setup, because once you get it working its great!
  • @Christian_Simon

    Gah, I thought it was all working but it is not...

    For the player who started the room, they can see everyone and everything is synched up great. For each additional person who joins the room, they can only see themselves, and anyone who joined after them. I have tried a couple of different things to try and fix this, but I can't seem to figure out why it isn't working, the following is what I ended up with but still isn't working fully.

    First I noticed that additional players spawn themselves fine, but they never spawn the previous players who were already in the room. What I then tried to do was add the following function to my "AvatarSpawner" script:
    public void InstantiateCurrentPlayers()
    {
    for (int i = 0; i < PhotonNetwork.otherPlayers.Length; i++)
    {
    GameObject spawnPoint = spawnPoints[i];

    GameObject.Instantiate(remotePlayerPrefab, spawnPoint.transform);
    }
    }
    That function is called from my Network Manager with the OnJoinRoom() via UnityEvent. This seems to work fine, however the Remote Avatar it spawns does not follow the HMD/Touch Controllers of the previous player, it just sits there and does nothing. Any Ideas?
  • For each additional person who joins the room, they can only see themselves, and anyone who joined after them.


    This sounds like a problem with event caching. Do you cache the Manual Instantiation events? If not, please do this by using the following code snippet or something similar which has the same effect and apply this setting when you call RaiseEvent:

    new RaiseEventOptions() { CachingOption = EventCaching.AddToRoomCache, Receivers = ReceiverGroup.All }

    This seems to work fine, however the Remote Avatar it spawns does not follow the HMD/Touch Controllers of the previous player, it just sits there and does nothing.


    You don't apply any ViewId to the remote player prefab. When following the guide, after the player has joined the room, he allocates a ViewId by using PhotonNetwork.AllocateViewID(); and shares that one with any other client. This is mentioned the Instantiating Avatars section of the guide. When another client receives the Instantiation event (the event contains the allocated ViewId) the client applies this ViewId to the attached PhotonView component in order to make it able to be synchronized.

  • I was caching the event like this:
    PhotonNetwork.RaiseEvent(InstantiateVrAvatarEventCode, viewID, true, new RaiseEventOptions() { CachingOption = EventCaching.AddToRoomCache, Receivers = ReceiverGroup.All });
    Which did not work for each successive person, only the first person would have their RaiseEvent passed along to the other players, no idea why.

    I ended up adding the "OnPhotonPlayerConnected()" to my spawning script, and then I just called that same function again to get it to actually propagate to all users including the newest one.

    This is what I ended up with, and it works so I moved on to other things:
    using UnityEngine;
    using System.Collections.Generic;

    public class TestSpawnManager : Photon.MonoBehaviour
    {
    public GameObject playerPrefab;
    public GameObject localAvatarPrefab;
    public GameObject remoteAvatarPrefab;

    public List< GameObject > spawnPoints = new List< GameObject >();

    public readonly byte InstantiateVrAvatarEventCode = 123;

    private GameObject localPlayer = null;
    private GameObject localAvatar = null;
    private List remoteAvatars = new List();

    private int viewID = 0;

    public void Start()
    {
    PhotonNetwork.OnEventCall += this.OnEvent;
    }

    public void InitializePlayer()
    {
    int spawnPointID = PhotonNetwork.otherPlayers.Length;
    localPlayer = GameObject.Instantiate(Resources.Load(playerPrefab.name), spawnPoints[spawnPointID].transform) as GameObject;
    localPlayer.transform.localPosition = Vector3.zero;
    localPlayer.transform.rotation = spawnPoints[spawnPointID].transform.rotation;
    localPlayer.transform.localScale = Vector3.one;

    SpawnPlayerAvatarOnNetwork();
    }

    private void SpawnPlayerAvatarOnNetwork()
    {
    viewID = PhotonNetwork.AllocateViewID();

    PhotonNetwork.RaiseEvent(InstantiateVrAvatarEventCode, viewID, true, new RaiseEventOptions() { CachingOption = EventCaching.AddToRoomCache, Receivers = ReceiverGroup.All });
    }


    public void OnPhotonPlayerConnected(PhotonPlayer player)
    {
    PhotonNetwork.RaiseEvent(InstantiateVrAvatarEventCode, viewID, true, new RaiseEventOptions() { CachingOption = EventCaching.AddToRoomCache, Receivers = ReceiverGroup.All });
    }

    private void OnEvent(byte eventcode, object content, int senderid)
    {
    if (eventcode == InstantiateVrAvatarEventCode)
    {
    GameObject newAvatar = null;

    if (PhotonNetwork.player.ID == senderid)
    {
    if(localAvatar == null)
    {
    newAvatar = Instantiate(Resources.Load(localAvatarPrefab.name), localPlayer.transform) as GameObject;
    localAvatar = newAvatar;
    }
    }
    else
    {
    bool spawnRemoteAvatar = true;

    foreach(RemotePlayerAvatarData remoteAvatarData in remoteAvatars)
    {
    if (remoteAvatarData.remotePlayerID == senderid)
    {
    spawnRemoteAvatar = false;
    }
    }

    if(spawnRemoteAvatar)
    {
    newAvatar = Instantiate(Resources.Load(remoteAvatarPrefab.name)) as GameObject;

    RemotePlayerAvatarData newRemoteAvatar = new RemotePlayerAvatarData();
    newRemoteAvatar.remotePlayerID = senderid;
    newRemoteAvatar.remoteAvatar = newAvatar;

    remoteAvatars.Add(newRemoteAvatar);
    }
    }

    if (newAvatar != null)
    {
    PhotonView pView = newAvatar.GetComponent();

    if (pView != null)
    {
    pView.viewID = (int)content;
    }
    }
    }
    }
    }

    [System.Serializable]
    public class RemotePlayerAvatarData
    {
    public int remotePlayerID = 0;
    public GameObject remoteAvatar = null;
    }

    My Initialize Player function is called from my Network Manager via UnityEvent when the user successfully joins a room. Then I allocate the ID and save it to a variable, and then I raise the spawn event. This takes care of giving all the already existing players the notification to spawn the new player.

    Then when a new player joins the room, all existing players call that SpawnEvent using the viewID that each of them has saved to a variable.

    In my OnEvent() I then handle making sure I dont spawn the local avatar again, and I make sure I don't spawn any remote avatars that I have not already spawned.

    I am sure its not the way its supposed to be done, but it works and I was getting tired of beating my head against it trying to get the RaiseEvent function to work as it is supposed to with how I understood the documentation.
  • Which did not work for each successive person, only the first person would have their RaiseEvent passed along to the other players, no idea why.


    Need to take a look at that when being back in office.

    Then when a new player joins the room, all existing players call that SpawnEvent using the viewID that each of them has saved to a variable.


    The problem here is that you cache this event (again) so when a new player joins afterwards he will receive this event multiple times - basically the more clients there are the more events are being received. Since you already worked around this problem and it is working for you, it is okay for now. As said I'm going to have a look at the event caching when being back in office.
  • Hi @SirisDracken,

    Which did not work for each successive person, only the first person would have their RaiseEvent passed along to the other players, no idea why.


    I have checked this again but I honestly can't reproduce this behaviour as the event caching is working for me. Please check if each client receives the event by removing the instantiation code and instead adding some Debug Logs to see if this really is a problem with cached events (if you still have this issue).
  • Hi @Christian_Simon

    I followed the Tutorial on the website and at the end I get the same problem as @SirisDracken .

    The thing is that the players who Instantiate later can not be notified about the players who were created before.
    When the player instantiates itself it sends an event to all the existing players and they instantiate the remote clients but the thing is that the player who has just instantiated doesn't see any other client.

    Can you please check that out? I don't want to do tricks to make it work.

    I can also share the code
    Her is the PhotonAvatarView.cs
    using System.Collections.Generic;
    using System.IO;
    using UnityEngine;
    
    namespace Source
    {
        public class PhotonAvatarView : MonoBehaviour
        {
            private PhotonView photonView;
            private OvrAvatar ovrAvatar;
            private OvrAvatarRemoteDriver remoteDriver;
            private int localSequence;
            
            private List<byte[]> packetData;
            
            public void Start()
            {
                photonView = GetComponent<PhotonView>();
                if (photonView.isMine)
                {
                    ovrAvatar = GetComponent<OvrAvatar>();
                    packetData = new List<byte[]>();
                    ovrAvatar.RecordPackets = true;
                    ovrAvatar.PacketRecorded += OnLocalAvatarPacketRecorded;
                }
                else
                {
                    remoteDriver = GetComponent<OvrAvatarRemoteDriver>();
                }
            }
            
    //        public void OnEnable()
    //        {
    //            if (photonView.isMine)
    //            {
    //                ovrAvatar.RecordPackets = true;
    //                ovrAvatar.PacketRecorded += OnLocalAvatarPacketRecorded;
    //            }
    //        }
     
            public void OnDisable()
            {
                if (photonView.isMine && ovrAvatar != null)
                {
                    ovrAvatar.RecordPackets = false;
                    ovrAvatar.PacketRecorded -= OnLocalAvatarPacketRecorded;
                }
            }
            
            public void OnLocalAvatarPacketRecorded(object sender, OvrAvatar.PacketEventArgs args)
            {
                if (!PhotonNetwork.inRoom || (PhotonNetwork.room.PlayerCount < 2))
                {
                    return;
                }
     
                using (MemoryStream outputStream = new MemoryStream())
                {
                    BinaryWriter writer = new BinaryWriter(outputStream);
     
                    var size = Oculus.Avatar.CAPI.ovrAvatarPacket_GetSize(args.Packet.ovrNativePacket);
                    byte[] data = new byte[size];
                    Oculus.Avatar.CAPI.ovrAvatarPacket_Write(args.Packet.ovrNativePacket, size, data);
     
                    writer.Write(localSequence++);
                    writer.Write(size);
                    writer.Write(data);
     
                    packetData.Add(outputStream.ToArray());
                }
            }
            
            private void DeserializeAndQueuePacketData(byte[] data)
            {
                using (MemoryStream inputStream = new MemoryStream(data))
                {
                    BinaryReader reader = new BinaryReader(inputStream);
                    int remoteSequence = reader.ReadInt32();
     
                    int size = reader.ReadInt32();
                    byte[] sdkData = reader.ReadBytes(size);
     
                    System.IntPtr packet = Oculus.Avatar.CAPI.ovrAvatarPacket_Read((System.UInt32)data.Length, sdkData);
                    remoteDriver.QueuePacket(remoteSequence, new OvrAvatarPacket { ovrNativePacket = packet });
                }
            }
            
            public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info)
            {
                if (stream.isWriting)
                {
                    if (packetData == null || packetData.Count == 0)
                    {
                        return;
                    }
     
                    stream.SendNext(packetData.Count);
     
                    foreach (byte[] b in packetData)
                    {
                        stream.SendNext(b);
                    }
     
                    packetData.Clear();
                }
     
                if (stream.isReading)
                {
                    int num = (int)stream.ReceiveNext();
     
                    for (int counter = 0; counter < num; ++counter)
                    {
                        byte[] data = (byte[])stream.ReceiveNext();
     
                        DeserializeAndQueuePacketData(data);
                    }
                }
            }
        }
    }

    And here is my networking scrip
    private void Start()
            {
                if (localPayerPrefab != null && remotePlayerPrefab != null) 
                {
                    int viewId = PhotonNetwork.AllocateViewID();
     
                    PhotonNetwork.RaiseEvent(Constant.INSTANTIATE_VR_AVATAR_EVENT_CODE, viewId, true, new RaiseEventOptions() { CachingOption = EventCaching.AddToRoomCache, Receivers = ReceiverGroup.All });
                }
            }
            
            private void OnEvent(byte eventcode, object content, int senderid)
            {
                if (eventcode == Constant.INSTANTIATE_VR_AVATAR_EVENT_CODE)
                {
                    GameObject go = null;
    
                    if (PhotonNetwork.player.ID == senderid)
                    {
                        go = Instantiate(localPayerPrefab);
                    }
                    else
                    {
                        go = Instantiate(remotePlayerPrefab);
                    }
                    
                    if (go != null)
                    {
                        PhotonView pView = go.GetComponent<PhotonView>();
     
                        if (pView != null)
                        {
                            pView.viewID = (int)content;
                        }
                    }
                    
                    InputTracking.Recenter();
                }
            }
            
            public void OnEnable()
            {
                PhotonNetwork.OnEventCall += OnEvent;
            }
     
            public void OnDisable()
            {
                //TODO remove player if he did leave the room
                PhotonNetwork.OnEventCall -= OnEvent;
            }

    It would be better if you could share a sample project or try to get the same problem on your side.
  • Hi @araratthehero,

    I guess I can have a look at this again next week. For now can you confirm, that the OnEvent function from the second code snippet actually gets called? You can easily do this by adding Debug.Log calls to it, which can print the received event code to the console. If those logs are showing your used event code, there is another problem which has to be identified. If your event codes don't get printed to the console, you might have a problem, where you have already received the instantiation events (you will receive them as soon as the client has joined the room), but don't have a listener registered to the OnEvent callback, at least none who can handle this certain event. Don't get me wrong: the code snippets looks fine so far, though it can be a 'timing' problem.
  • araratthehero
    edited May 2018
    Good afternoon @Christian_Simon

    I can also debug my code so I've tried that already the thing is that OnEvent function is called when I Raise the event on JoinedRoom function and the avatar is created. It is also called when a new player is added to the room and every time as a new player joins to the room I see the OnEvent function fire. The case here is that the player who just joined the hass the event fired only once to create himself and the event is not fired to let him know that also other players are in the room. But players who are already in the room get the event called and can see the new player. The case only exists on new players who join and can not see objects and players which are instantiated before.

    If you can also share a sample working project which I can test on my end also can be really nice.
  • The case here is that the player who just joined the hass the event fired only once to create himself and the event is not fired to let him know that also other players are in the room.


    This is the problem I have mentioned before: you don't have an OnEvent listener that can handle the instantiation calls at the time they are received. It works for the instantiation call of local client because OnEnable() is called before Start() where you actually raise the event for the local client. To get this working you need an OnEvent handler which is basically available (marked as DontDestroyOnLoad) in both / all scenes.
  • My issue is fixed.

    The problem was from my networking class. When the client joins the room I open a new level and Instantiate the player. When the next player joins the room he gets the event but I did not have a separate handler to catch the events on networking class. What I did was to create a few methods and allow other classes to add listeners and get the callbacks fired when a new event is received from the networking class. But when the networking class gets the events it do cache locally so everyone else can ask for their events and get them correspondingly. I did really dig into this issue really deep. If anyone face this kind of problems write me here or personally and I can share what I learned from here.

    And of course thanks to @Christian_Simon for fast responses and @SirisDracken to mention all of the issues related to the topic.
  • Hi @araratthehero and @SirisDracken , are either of you developing for the Go or just the Rift? I'm having an issue with sending Avatar packets - it's crashing whichever client joins the room next after the first person who creates the room - and I think it's because it's on the Go specifically.
  • Atley
    Atley
    edited February 2019
    Hi @araratthehero, would you mind sharing your event handler methods? Thank you!