Syncing Voice Chat with Oculus Avatar

Hey, everyone! I built a game using the instructions on how to sync Oculus Avatars through Photon (the documentation was taken down from Photon's website afterwards, so I can't link it). I was wondering if there was a way for me to sync voice chat over as well? I have attached a recorder, speaker, and voice view component on both the Local and Remote Avatar prefabs, yet it doesn't sync any audio. I have initialized the recorder through script, and don't have any idea as what to do! Please help!

Comments

  • JohnTube
    JohnTube ✭✭✭✭✭
    edited April 2020
    Hi @CompuGeniusPrograms,

    Could you do some logging / debugging with hints from this discussion?
  • JohnTube wrote: »
    Hi @CompuGeniusPrograms,

    Could you do some logging / debugging with hints from this discussion?

    I'll try! I did so much searching, yet I didn't find that thread, lol.
  • CompuGeniusPrograms
    edited April 2020
    JohnTube wrote: »
    Hi @CompuGeniusPrograms,

    Could you do some logging / debugging with hints from this discussion?

    I'm a bit confused as to how to log it, tbh. I have my PhotonVoiceNetwork, PhotonVoiceView, Recorder, and Speaker all on ERROR, yet nothing logs.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @CompuGeniusPrograms,

    Increase logging level to Warning or more.
    Manually log the relevant PhotonVoiceView as shown in that linked discussion.

    Screenshot interpretation:
    Not sure if Editor is player 1 or 4.
    Not sure why a player has two voices, do you have two Recorder component working at the same time on a single client? is this expected, on purpose?
    If the Unity Editor is player 4 then you successfully linked local Speaker for voice 2 of the remote player 1.

    There is also an Oculus option that needs to be disabled, see here.
  • JohnTube wrote: »
    Hi @CompuGeniusPrograms,
    Increase logging level to Warning or more.
    Manually log the relevant PhotonVoiceView as shown in that linked discussion.
    Lol, I thought ERROR logging was above WARNING.
  • JohnTube wrote: »
    There is also an Oculus option that needs to be disabled, see here.
    OMG! This fixed it! Thanks so much! I will be releasing tutorials on my Youtube channel on how to make a multiplayer Oculus game with Photon sometime soon, and hopefully it will help future developers!
  • CompuGeniusPrograms
    edited May 2020
    @JohnTube Hi! So I changed my game up a bit, and now I get this error when trying to use voice chat:
    [AudioManager.PhotonVoiceNetwork] Unexpected: VoiceInfo.UserData should be int/ViewId, received: null
    UnityEngine.Debug:LogErrorFormat(Object, String, Object[])
    Photon.Voice.Unity.VoiceLogger:LogError(String, Object[]) (at Assets/Photon/PhotonVoice/Code/VoiceLogger.cs:67)
    Photon.Voice.PUN.PhotonVoiceNetwork:CheckLateLinking(PhotonVoiceView, Int32) (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceNetwork.cs:432)
    Photon.Voice.PUN.PhotonVoiceView:CheckLateLinking() (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceView.cs:145)
    Photon.Voice.PUN.PhotonVoiceView:Init() (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceView.cs:321)
    Photon.Voice.PUN.PhotonVoiceView:Start() (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceView.cs:138)
    
    What do I do?
  • @JohnTube Hi! So I changed my game up a bit, and now I get this error when trying to use voice chat:
    [AudioManager.PhotonVoiceNetwork] Unexpected: VoiceInfo.UserData should be int/ViewId, received: null
    UnityEngine.Debug:LogErrorFormat(Object, String, Object[])
    Photon.Voice.Unity.VoiceLogger:LogError(String, Object[]) (at Assets/Photon/PhotonVoice/Code/VoiceLogger.cs:67)
    Photon.Voice.PUN.PhotonVoiceNetwork:CheckLateLinking(PhotonVoiceView, Int32) (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceNetwork.cs:432)
    Photon.Voice.PUN.PhotonVoiceView:CheckLateLinking() (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceView.cs:145)
    Photon.Voice.PUN.PhotonVoiceView:Init() (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceView.cs:321)
    Photon.Voice.PUN.PhotonVoiceView:Start() (at Assets/Photon/PhotonVoice/Code/PUN/PhotonVoiceView.cs:138)
    
    What do I do?

    Ok, so I don't know what the culprit was, but I fixed it by waiting a second after initializing the Recorders.
  • Ok, so I don't know what the culprit was, but I fixed it by waiting a second after initializing the Recorders.
    Well, the error's back after not changing a thing! Someone, please tell me what causes the error, so I can fix it!
  • Ok, so I don't know what the culprit was, but I fixed it by waiting a second after initializing the Recorders.
    Well, the error's back after not changing a thing! Someone, please tell me what causes the error, so I can fix it!

    It seems that the error only occurs if the editor joined afterwards.
  • Ok, so I don't know what the culprit was, but I fixed it by waiting a second after initializing the Recorders.
    Well, the error's back after not changing a thing! Someone, please tell me what causes the error, so I can fix it!

    I think the error is caused when the speaker is instantiated/linked before the recorder is instantiated/initialized (not sure in either case). The way I fixed this, was I just made sure the LocalAvatar (recorder) was instantiated, and that it was initialized, before instantiating the RemoteAvatar (speaker).
  • OMFG! I can't stand it! The error comes and goes every other run! It's impossible! Please @JohnTube or anyone else at Photon, please help!
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @CompuGeniusPrograms,

    I will look into this now, for the time being, we rely on your patience and understanding.

    If it's a timing issue we will try to fix it.
  • @JohnTube I think I fixed it for good. It seems I needed to wait a few seconds between connecting the speaker, and continuing the rest of the program. I don't know what caused it, but I think I fixed it.
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited May 2020
    Hi @CompuGeniusPrograms,

    That error log will be a warning in the future and the text will be changed a bit.
    So it may be harmless.

    However, I still want to understand the issue here.
    I need your help to get a repro case or minimal repro steps and how you fixed it.

    I'm guessing you use manual PUN instantiation and two prefab variants: local and remote.
    When do you instantiate?
    What am I missing?
  • I followed the old Photon instructions for instantiated Oculus Avatars. It has been taken down since then. I then incorporated delayed loadout, which means I'm instantiating the local avatar after the remote avatar (I think). That's what caused the error, I believe.
  • CompuGeniusPrograms
    edited May 2020
    I followed the old Photon instructions for instantiated Oculus Avatars. It has been taken down since then. I then incorporated delayed loadout, which means I'm instantiating the local avatar after the remote avatar (I think). That's what caused the error, I believe.
    private void GetLoggedInUserCallback(Message<User> msg)
        {
            localAvatar = Instantiate(Resources.Load("LAvatar")) as GameObject;
            localAvatar.GetComponent<OvrAvatar>().oculusUserID = msg.Data.ID.ToString();
    
            audioManager.recorder = localAvatar.GetComponent<Recorder>();
    
            PhotonView photonView = localAvatar.GetComponent<PhotonView>();
    
            if (PhotonNetwork.AllocateViewID(photonView))
            {
                RaiseEventOptions raiseEventOptions = new RaiseEventOptions
                {
                    CachingOption = EventCaching.AddToRoomCache,
                    Receivers = ReceiverGroup.Others
                };
    
                object[] content = new object[] {
                    photonView.ViewID,
                    msg.Data.ID.ToString()
                };
    
                PhotonNetwork.RaiseEvent(InstantiateVrAvatarEventCode, content, raiseEventOptions, SendOptions.SendReliable);
            }
        }
    
        void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
    
                StartCoroutine(InitializeSpeaker());
            }
        }
    
        IEnumerator InitializeSpeaker()
        {
            audioManager.speaker = remoteAvatar.GetComponent<Speaker>();
    
            yield return new WaitForSeconds(3); // THE "FIX"!
    
            audioManager.speaker.GetComponent<PhotonVoiceView>().RecorderInUse = audioManager.recorder;
            audioManager.recorder.GetComponent<PhotonVoiceView>().SpeakerInUse = audioManager.speaker;
        }
    

    @JohnTube Here's my full avatar instantiation code, along with the "fix". If more explanation is necessary, I don't mind doing so.
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited May 2020
    Hi @CompuGeniusPrograms,

    Thanks for your help!

    First, you do not need a Recorder for the remote avatar but only Speaker.
    Second, if you enable RoomOptions.PublishUserId and set UserId to Oculus UserId then Photon will automatically synchronize UserIDs in the room for you via Player.UserId.

    Could you share with us the PhotonVoiceView settings?
    Why don't you assign Recorder and Speaker already.

    What happens if you change the code in each of the following cases:

    A. Do not do any explicit initialization.
    void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
            }
        }
    

    B. Call Init only.
    void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
    
                audioManager.speaker.GetComponent<PhotonVoiceView>().Init();
            }
        }
    

    C. Wait for end of frame then Init.
        void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
    
                StartCoroutine(InitializeSpeaker());
            }
        }
    
        IEnumerator InitializeSpeaker()
        {
            yield return new WaitForEndOfFrame();
    
            audioManager.speaker.GetComponent<PhotonVoiceView>().Init();
        }
    
  • JohnTube wrote: »
    Hi @CompuGeniusPrograms,
    First, you do not need a Recorder for the remote avatar but only Speaker.

    I don't have a Recorder on my Remote Avatar. I have it on my Local Avatar.
    JohnTube wrote: »
    Hi @CompuGeniusPrograms,
    Second, if you enable RoomOptions.PublishUserId and set UserId to Oculus UserId then Photon will automatically synchronize UserIDs in the room for you via Player.UserId.

    What do I need that for?


    JohnTube wrote: »
    Hi @CompuGeniusPrograms,
    A. Do not do any explicit initialization.
    void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
            }
        }
    

    B. Call Init only.
    void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
    
                audioManager.speaker.GetComponent<PhotonVoiceView>().Init();
            }
        }
    

    C. Wait for end of frame then Init.
        void IOnEventCallback.OnEvent(EventData photonEvent)
        {
            if (photonEvent.Code == InstantiateVrAvatarEventCode)
            {
                remoteAvatarReady = true;
            }
    
            if (remoteAvatarReady && audioManager.recorder && !instantiatedRemoteAvatar)
            {
                remoteAvatar = Instantiate(Resources.Load("RemoteAvatar")) as GameObject;
    
                PhotonView photonView = remoteAvatar.GetComponent<PhotonView>();
                object[] customData = (object[])photonEvent.CustomData;
                photonView.ViewID = (int)customData[0];
                remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
                instantiatedRemoteAvatar = true;
    
                StartCoroutine(InitializeSpeaker());
            }
        }
    
        IEnumerator InitializeSpeaker()
        {
            yield return new WaitForEndOfFrame();
    
            audioManager.speaker.GetComponent<PhotonVoiceView>().Init();
        }
    

    I'm confused by your many code blocks.
  • @JohnTube I still can't seem to fix this issue. I understand it's harmless, but i would like to get to the bottom of it.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @CompuGeniusPrograms,
    I'm confused by your many code blocks.
    The purpose of those code blocks is to try them one by one to see if any one resolves the issue without the need to reassign Speaker and Recorder like you do in your InitializeSpeaker.
    I don't have a Recorder on my Remote Avatar. I have it on my Local Avatar.
    what about this then?
    audioManager.speaker.GetComponent<PhotonVoiceView>().RecorderInUse = audioManager.recorder;
    
    What do I need that for?
    to avoid sending UserId as custom instantiation data.
    this
    object[] content = new object[] {
                    photonView.ViewID,
                    msg.Data.ID.ToString() // <---
                };
    
    and this:
    remoteAvatar.GetComponent<OvrAvatar>().oculusUserID = (string)customData[1];
    
    but it looks like somehow you "log in" to Oculus after you join a Photon room?
    unlike what we do in custom authentication here. But that's another topic for another discussion and it is not a problem if it works the way you want it.
    btw if you missed it (nothing wrong with it, just my OCD):
    audioManager.speaker.GetComponent<PhotonVoiceView>().RecorderInUse = audioManager.recorder;
    audioManager.recorder.GetComponent<PhotonVoiceView>().SpeakerInUse = audioManager.speaker;
    
    *.speaker.*.RecorderInUse = *.recorder;
    *.recorder.*.SpeakerInUse = *.speaker;
    You did not answer this:
    Could you share with us the PhotonVoiceView settings?
    Why don't you assign Recorder and Speaker already.
    Two screenshots of PhotonVoiceView on each prefab variant would help.
  • CompuGeniusPrograms
    edited May 2020
    JohnTube wrote: »
    Hi @CompuGeniusPrograms,
    I'm confused by your many code blocks.
    The purpose of those code blocks is to try them one by one to see if any one resolves the issue without the need to reassign Speaker and Recorder like you do in your InitializeSpeaker.
    I don't have a Recorder on my Remote Avatar. I have it on my Local Avatar.
    what about this then?

    I was setting the audioManager.speaker to the speaker on the RemoteAvatar.

    I removed those lines anyway.
    JohnTube wrote: »
    You did not answer this:

    https://imgur.com/5CRLgh7
    https://imgur.com/y5zsYsQ
  • JohnTube wrote: »
    Hi @CompuGeniusPrograms,
    That error log will be a warning in the future and the text will be changed a bit.
    So it may be harmless.

    I'm so fed up with this garbage logging that Photon does. I just edited the script to log a warning instead of an error. When Photon gets their act together, and fixes it, please let me know.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Please test again after importing this unitypackage into Photon Voice 2.16.1.
  • JohnTube wrote: »
    Please test again after importing this unitypackage into Photon Voice 2.16.1.

    I'm about to test. If I may know, what different in this version?
  • JohnTube wrote: »
    Please test again after importing this unitypackage into Photon Voice 2.16.1.

    The "error" still occurs.
  • JohnTube
    JohnTube ✭✭✭✭✭
    closing, continue here.
This discussion has been closed.