Unable to hear voice across Oculus Quests or Oculus Quest and PC-Unity

Aitor
Aitor
edited April 2020 in Photon Voice
Hi,

I am using both PUN2 and Photon Voice to create a multiplayer unity experience.

My setup consists of a Lobby Scene, where the players automatically join or create a room and then load a new Unity Scene, containing the multiplayer environment.

The lobby scene creates the connection following the tutorials and finally the masterClient, during the OnJoinedRoom() function call, calls
- PhotonNetwork.LoadLevel("LR Tanker");

My LR Tanker Scene has a "Voice" gameObject, pretty much copied from DemoVoicePun containing the following components:
* Photon Voice Network
- AutoConnect And Join = true;
- AutoLeave And Disconnect = true;
- Create Speaker if Not Found = true;
- DDOL = true;
- Primary Recorder = "Voice (Recorder)" (the voice component in the same gameObject)

* Recorder
- Transmit Enabled = true;
- Auto Start = true;
- Debug Echo = true;

My Player Prefab is a OVR Dummy made up of 3 networked individual objects, Left and Right hand, both with a photon view component and a Head with:
- Photon View
- Audio Source
- Photon Voice View
- Recorder
- Speaker
- Mic Amplifier
replicating the components appearing in the ZomBear, ZomBunny, ... of the Voice Demo.

I instantiate the object as such:
- GameObject PLM = PhotonNetwork.Instantiate("NetworkedPlayerMaster", spawnPos, Quaternion.identity, 0);

I am logging the following fields in my instantiated prefabs (clients)
S
tart()
{
        photonVoiceView = GetComponent<PhotonVoiceView>();
        punVoiceNetwork = PhotonVoiceNetwork.Instance;
        recorder = Voice.GetComponent<Recorder>();

        PhotonView pv = GetComponent<PhotonView>();

        photonVoiceView.Init();
        recorder.StopRecording();
        recorder.StartRecording();

        Debug.LogWarningFormat("PhotonView ViewId {0}", pv.ViewID);

        Debug.LogWarningFormat("PunVoiceNetwork isActiveAndEnabled {0}", punVoiceNetwork.isActiveAndEnabled);
        Debug.LogWarningFormat("PunVoiceNetwork clientState {0}", punVoiceNetwork.ClientState);

        Debug.LogWarningFormat("PhotonVoiceView isSetup {0}", photonVoiceView.IsSetup);
        Debug.LogWarningFormat("PhotonVoiceView isRecorder {0}", photonVoiceView.IsRecorder);
        Debug.LogWarningFormat("PhotonVoiceView isSpeaker {0}", photonVoiceView.IsSpeaker);

        Debug.LogWarningFormat("Recorder IsInitialized {0}", recorder.IsInitialized);
        Debug.LogWarningFormat("Recorder IsRecording {0}", recorder.IsRecording);
        Debug.LogWarningFormat("Recorder TransmitEnabled {0}", recorder.TransmitEnabled);
        Debug.LogWarningFormat("Recorder IsCurrentlyTransmitting {0}", recorder.IsCurrentlyTransmitting);

        Debug.LogWarningFormat("PhotonVoiceView isSpeakerLinked {0}", photonVoiceView.SpeakerInUse.IsLinked);

        Speaker sp = GetComponent<Speaker>();
        Debug.LogWarningFormat("Speaker isLinked {0}", sp.IsLinked);
}
Also the same Logs in the Update method, except for the Init() and Stop/Start Recording calls.
Once the "clientState" is "joined" my logs read the following.

PhotonView ViewId 1002

PunVoiceNetwork isActiveAndEnabled True
PunVoiceNetwork clientState Joined

PhotonVoiceView isSetup True
PhotonVoiceView isRecorder False
PhotonVoiceView isSpeaker True

Recorder IsInitialized True
Recorder IsRecording True
Recorder TransmitEnabled True
Recorder IsCurrentlyTransmitting False

PhotonVoiceView isSpeakerLinked False
Speaker isLinked False


At best I hear about one second of audio via the Debug Echo, at the beginning, which then loops indefinitely.
I cannot hear audio across devices.

I have tried multiple things and I am running out of ideas. Some help would be appreciated.

Comments

  • I have made another test where there were 3 users connected using Oculus Quest and one more instance running on the Editor.
    The log makes sense to me, but still there was no voice being carried over.

    The PhotonView Running on the editor is ID 4001


    PhotonView ViewId 4001
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder true
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked true
    Speaker isLinked true
    ****************************************************************************************************

    PhotonView ViewId 1002
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder False
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked False
    Speaker isLinked False

    ****************************************************************************************************

    PhotonView ViewId 2001
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder False
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked False
    Speaker isLinked False

    ****************************************************************************************************

    PhotonView ViewId 3001
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder False
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked False
    Speaker isLinked False
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @Aitor,

    Thank you for choosing Photon and for your detailed report!

    The issue is clear, remote voice streams are not being linked to the local Speaker components through PhotonVoiceView for some reason. The linking happens using PhotonViewId so these should match between local and remote clients. Also 'timing' sometimes is the reason why linking does not happen. You could manually try to force linking using PhotonVoiceView.Init() for all remote avatars (when PhotonVoiceView.IsSpeaker = true).

    If you increase logs (at least warning level) of PhotonVoiceView, PhotonVoiceNetwork and Speaker you could see something about this, why the linking fails.

    From the Unity Editor check the "Cached Remote Voices' Info" in PhotonVoiceNetwork's inspector, you should see at least 3 entries there.
  • Hi @JohnTube,

    Thank you for your quick reply.

    PhotonVoiceView.Init() was already being called on clients.

    I increased the log level to warning but I cannot see any log about failing links but I have seen something odd in the Cached Remove Voices' Info.
    https://photos.app.goo.gl/fJYWrYnvH9wBRHRt5

    I feel like voice # should match the player # (this test was using Unity Editor and one Oculus Quest, with the later as master)

    I hope you can point me in the right direction.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @Aitor,

    According to the screenshot, Photon Voice 2 is properly working!

    The Speaker components are linked as you can see.
    You could double check using your logging statements.

    So the linking was not the issue or it's not anymore.

    Check audio input device or maybe stream an AudioClip to test or change microphone type (Unity vs. Photon) or change microphone device.
    Check audio output device or see Speaker component Inspector.
    Check AudioSource 3D settings, disable them via spatialBlend = 0.

    Make sure Recorder is transmitting and recording.
    I feel like voice # should match the player #
    No not necessarily, each player can have up to 255 concurrent voice streams, on each voice stream creation voice # is incremented.
  • Aitor
    Aitor
    edited April 2020
    Hi, @JohnTube

    To be honest I am confused as to what might be causing the problem then.
    So far none of the proposed solutions fixed the problem. I think I have made some progress though.

    In my latest test, with 2 Oculus Quest devices and another instance running on the Editor, I have managed to hear the MasterClient from all devices (including itself if Debug Echo was enabled).
    The regular clients only capture the first second of sound, which then plays in a loop for all connected clients.

    My first connected client will create the room, the rest will join. This is done from a lobby Scene containing a game object with the PhotonVoiceNetwork component you saw in my previous post and a Recorder that is DontDestroyOnLoad.

    From that point all clients load the actual Unity Scene and the player prefabs are instatiated by another DDOL sceneManager script (if they have not been instantiated already) during Start().

    Logs of the created instances, the client running in the editor is ID 3001 and the master client is ID 1002

    PhotonView ViewId 3001
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder true
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked False
    Speaker isLinked False

    PhotonView ViewId 1002
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder False
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked true
    Speaker isLinked true

    PhotonView ViewId 2001
    PunVoiceNetwork isActiveAndEnabled true
    PunVoiceNetwork clientState Joined
    PhotonVoiceView isSetup true
    PhotonVoiceView isRecorder False
    PhotonVoiceView isSpeaker true
    Recorder IsInitialized true
    Recorder IsRecording true
    Recorder TransmitEnabled true
    Recorder IsCurrentlyTransmitting False
    PhotonVoiceView isSpeakerLinked true
    Speaker isLinked true

    I tried changing microphone type and it made no difference.

    If I tap on the mic for any of the clients while they are connecting, the first second or so of audio gets picked up, as I mentioned, also Recorder.IsCurrentlyTransmitting changes to true.

    Something I don't have 100% clear is, Should I have only one recorder component in my "Voice" object, or should each instatiated player have their own recorder? If the later, which do I set as primary recorder for my Photon Voice Network component?

    I appreciate all your help, I can provide other logs if necessary.
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited April 2020
    Hi @Aitor,

    As far as I can see from previous screenshot and last logs, Photon Voice should be working as expected.
    There could be some recording or playback issues.
    Otherwise there could be some network issues (loss, disconnects, etc.)

    Did you try any of my previous suggestions?
    Check audio input device or maybe stream an AudioClip to test or change microphone type (Unity vs. Photon) or change microphone device.
    Check audio output device or see Speaker component Inspector.
    Check AudioSource 3D settings, disable them via spatialBlend = 0.

    ---
    Something I don't have 100% clear is, Should I have only one recorder component in my "Voice" object, or should each instatiated player have their own recorder? If the later, which do I set as primary recorder for my Photon Voice Network component?
    Recorder component is needed only on the local client. You need at least one and in most cases you will use only one. Each Recorder is an outgoing voice stream (same player # and different voice #). So in theory you could have 256 Recorders all transmitting at the same time from a single Player. But as I said you most likely will need only one, that's why we have came up with the PrimaryRecorder. Which is there to remain and be used every time you join a room instead of having one per prefab (useless on remote character/avatar objects) and instantiated every time. The good thing when it remains in the scene (as opposed to prefab) is that it can persist/keep some useful settings, e.g. VAD calibration between sessions / rooms.
  • Hi @JohnTube , thanks for your reply.
    Check audio input device or maybe stream an AudioClip to test
    Audio input device is working fine, works in the demo scene and other apps.
    Tested with AudioClip, it plays on a loop (when "loop" is true) on all joined devices
    or change microphone type (Unity vs. Photon) or change microphone device.
    Already tested that, as mentioned in my previous response. It made no difference.
    Check audio output device or see Speaker component Inspector.
    Audio output device is working for other apps, please find below a screen cap of my speaker component, attached to my instantiated prefab and log level raised to warning as you suggested earlier in this thread.
    https://photos.app.goo.gl/z5F4TmPU1STN5Nvr8
    Check AudioSource 3D settings, disable them via spatialBlend = 0.
    spatialBlend was already 0

    Appreciate your explanation on the Recorder component, now it's a lot more clear.

    So far the issue is not resolved and I will keep testing things. Feel free to suggest other possible solutions.
    Thank you again.
  • Hi @JohnTube,

    I found the time to explore this issue further, I was reading the forums and saw this post and the answer linking to this solution.

    It did solve my problem too, so you can consider closing the thread.
    Also, I would suggest a big flashy warning on your tutorials to warn other people using Oculus or OVR prefabs of this problem. It can potentially save them a lot of time.

    Thank you a lot for your support and keep up the good work!
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @Aitor,

    We already added this to the documentation but the docs website deploy did not happen yet.
    Expect it to show up this week.

    Thank you for your suggestion and sorry we did not tell you about this sooner.
  • Hello,
    I would like to be pointed to the solution please. We're experiencing the same problem.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Please test again after importing this unitypackage into Photon Voice 2.16.1.
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited May 2020
    closing, continue here.
This discussion has been closed.