Leaving room and rejoining the same room audio issue

Good morning.

Let me start by explaining our situation. Two players join a lobby from the unity editor. Player 1 creates a room and this loads a scene. Player 2 selects that room from a list and hits join. Player 2 joins Player 1 in the scene and they can both speak and hear each other.

Player 2 then leaves the room and returns to the lobby. He once again selects the room from the list and hits join. Player 1 and Player 2 are both again in the same scene. Player 1 can hear Player 2. Player 2 cannot hear Player 1. This happens in the reverse as well, if Player 1 left the room and comes back.

However, if Player 1 and Player 2 join a room and then change the scene together, not going back to the lobby, audio continues to work correctly. It seems our issue seems to be returning to the lobby. We have an object in DoNotDestryOnLoad called VoicePre, that contains the scripts, "Recorder" and "Photon Voice Network." On Photon Voice Network, we have Auto Connect and Join TRUE and Auto Leave and Disconnect TRUE.

We are not sure if we are using Voice 2 in a way it was not intended or if we have an issue reconnecting the ability to hear the other person when we re-connect. We do not see any errors in the log and we have attempted to debug, but we cannot seem to locate where this is happening.

We are currently using Unity 2017.4.2f2 and Pun 2 v2.6 and installed Voice 2 v2.2. Any help here would be greatly appreciated.

-Ted Hartle

Comments

  • JohnTube
    JohnTube ✭✭✭✭✭
    edited January 2019
    Hi @TedHartle,

    Thank you for choosing Photon!

    Could you set log level to ALL for "Recorder" and "PhotonVoiceNetwork", reproduce then send us logs from the two players?

    Do you use the regular/default flow of the PUN2 - Voice2 integration with the audio prefab that has "PhotonVoiceView"?

    but we cannot seem to locate where this is happening.
    We thought we fixed this in 2.2, the "PhotonVoiceView" should do a later Speaker-Recorder binding always based on the "PhotonView"'s ViewId inside PhotonVoiceView.OnEnable specifically PhotonVoiceNetwork.Instance.CheckLateLinking(this, photonView.ViewID);

    It seems we fixed a late binding after join but not a late binding after (re)join!
    Does the actor keep the same actor number and view IDs when re joining? do you use rejoin or join when returning to the room from the lobby?
  • TedHartle
    TedHartle
    edited January 2019
    Thanks for the response, John.

    Yes, we used the tutorial link you sent in the last post when we originally set up this project, but thank you for the link. We followed it as best we could, however, this is our first multiplayer attempt, so we may have miss or misinterpreted a step that is causing out issue.

    We could not locate where the log information is saving out to upload it here. Currently, we are still working in the editor, we have not compiled an executable yet, if that is the step we are missing. Here are some of the warnings we saw in the console:

    [VoicePre(Clone).PhotonVoiceNetwork] No PhotonView with ID 2001 found. Remote voice 3/1 not linked.
    [VoicePre(Clone).PhotonVoiceNetwork] Speaker is null. Remote voice 3/1 not linked.

    These warnings pop up when player 2 rejoins the room from the lobby. Based off your previous response, we confirmed that in our project the actor does not keep the same actor number or view ID when they rejoin the room from the lobby. Player 2 has a viewer ID of 2001 and an actor number of 2 when they first join the room, audio works great. When Player 2 leaves the room to the lobby and rejoins the room, their viewer ID is 3001 and an actor number of 3.

    If you need to see a full log, let us know where it is usually saved by Unity. However, if by our description above, it seems clear the viewer ID and Actor Numbers are our issue, we would appreciate any additional direction on how to keep them the same or otherwise solve that issue.

    Thanks again!

    -Ted
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @TedHartle,

    No PhotonView with ID 2001 found. Remote voice 3/1 not linked.

    When Player 2 leaves the room to the lobby and rejoins the room, their viewer ID is 3001 and an actor number of 3.
    I'm trying to understand this.

    There is PUN's actor number and Voice actor number.
    PhotonView and ViewID are PUN exclusive
    Here I see that:

    Voice client received a remote voice info event due to a Recorder starting transmission of voice actor with actor number 3 and voice ID 1. (3/1 == {voice actor number} / {voice ID or "recorder ID"})...
    But the thing is, the received ViewID is 2001 which belongs to PUN actor number 2 (view ID n°1 for PUN actor number 2) but you are saying that after joining the room again those change to 3 and 3001.

    How do you join the room again? do you call PhotonNetwork.JoinRoom or PhotonNetwork.RejoinRoom?
    How do you leave rooms?
    What is the PlayerTTL value you set when you create rooms?
    Do you enable CleanupCacheOnLeave/DeleteCacheOnLeave when you create rooms?
  • TedHartle
    TedHartle
    edited January 2019
    Good afternoon, John.

    Let me start by answering your questions. Initially, until you mentioned it, we only used PhotonNetwork.JoinRoom. We leave using PhotonNetwork.LeaveRoom. Initially, we did not use any PlayerTTL. We looked into this and Rejoin and describe our efforts below. We did not previously use CleanupCacheOnLeave, but we do now, however, DeleteCacheOnLeave did not present itself as an available function in visual studio when we attempted to use it.

    Based off your posts, we have noticed what might be causing our issue. We also found a work around that allows us to leave and rejoin and have audio reconnect each time, however, as I am sure you will see, the solution is not ideal.

    First, a couple of observations as we attempted to resolve our issue with the information you provided:
    • PhotonNetwork.Rejoin requires PlayerTTL to function
    • PhotonNetwork.Join will not rejoin the same player to a room if there is a PlayerTTL, or at least not until the PlayerTTL duration has been reached. So when we use Join, we remove PlayerTTL.

    I mention those points in case someone else reads this post and attempts Rejoin as well.

    Unfortunately, rejoin did not seem to resolve our issue, we still had the same audio issues when player 2 left and returned. However, we investigated our scene more and found a work-a-round. When the function OnPlayerEnteredRoom is called, we run this custom co-routine:
    private IEnumerator ReInit(float delayTime)
    {
        recorder.MicrophoneType = Recorder.MicType.Photon;
        recorder.ReInit();
        yield return new WaitForSeconds(delayTime);
        recorder.MicrophoneType = Recorder.MicType.Unity;
        recorder.ReInit();
    }
    Basically, we found that when player 2 returned to the room, Player 1 could hear player 2, but player 2 cannot hear player 1. We found if player 1 toggled MicrophoneType as above, manually, and clicked Re-Init, it fixed the audio issue. If we switched positions and player 2 stayed in the scene and player 1 left and returned, this solution always fixed our issue. This workaround resolved our issue whether we used Rejoin or Join to come back into the room. In our workaround, we decided to remove PlayerTTL and Rejoin, so that players would join and leave instantaneously, rather than waiting for a timeout. But we are glad to know how to use Rejoin and PlayerTTL now, for future projects.

    Clearly, there is a better way, but we hope sharing this workaround will help you understand our issue and point us to a better solution.

    :: Edit :: Well, that's unfortunate. The above workaround was working in the Unity Editior, but we built it out and tested it, but it did not work. We had the same exact issue we have been having. Player 2 leaves and re-enters the room and neither player can hear each other. Same if Player 1 does the same thing. SO, back to the drawing board, I guess. We removed the ReInit workaround and built again, just to see if it would work "normally" and had the same issue. Any suggestions you have based off the above info is greatly appreciated.

    -Ted
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @TedHartle,

    Let's forget about PlayerTTL, Rejoin vs. Join and CleanupCacheOnLeave.
    I was thinking about this issue the opposite way: I thought the returning player cannot be heard by the other player...

    I still think this is an issue related to the timing of scene switching vs. events processing.

    When a player leaves, his audio prefab with PhotonVoiceView is destroyed right?
    And when he returns this audio prefab with PhotonVoiceVIew is instantiated again?

    I still need those full logs, Unity Editor's console logs are enough. You can select "Open Editor Log" in Unity’s Console window after clicking on the top right small button. I need a log file per player where you reproduce this issue. Of course after increasing log level for all voice components, even those in prefab.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @TedHartle,

    Could you import these updated scripts and try to reproduce again?
  • Thank you for the scripts. We ran a test and here are our logs from the Unity editor. Any thoughts?

    Player 1 Log

    Player 2 Log


    -Ted
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited January 2019
    Hi @Ted,

    So even with the updated scripts, the issue persists?
    Unfortunately, the logs do not show anything useful.
    I can see, in player1 logs that player2 joined as actor number 2 with PhotonVoiceView having ViewId 2001 and then left. In this first join, the remote voice info event was received and processed correctly (speaker linked) by player 1. After that, however, when player2 joins the room again with actor number 3, player1 does not receive the new remote voice info event...

    I will have to try and reproduce on my end.
    If you can have a minimal project (w/o anything VR) that would help.
  • Good morning,

    Sorry for the late delay. We got hit with a deadline for another project that we need to focus on. As soon as we can make a proxy project and send it your way, we will, but it might be a week or more. I will update here when we have a download link. Again, thanks for your support and we look forward to looking into this issue with you.

    -Ted
  • MartinD
    MartinD
    edited February 2019
    Hello Team,

    we are experiencing a similar problem with PUN Voice 2 (tested with Unity 2018 and above).
    Initiallly the voice chat is working fine for every player in a room.
    But if any one player leaves and rejoines a room he can not be heared by others.
    As it turns out the IAudioOut on the the Speaker in question gets not properly initialized when rejoining a room.

    The code in question is following (from Speaker.cs) -
    protected override void Awake()
            {
                base.Awake();
                Func<IAudioOut> factory = () => new AudioStreamPlayer(new VoiceLogger(this, "AudioStreamPlayer", this.LogLevel),  
                new UnityAudioOut(this.GetComponent<AudioSource>()), "PhotonVoiceSpeaker:", this.Logger.IsInfoEnabled); 
    ...

    In order to fix this issue i put the inline/lambda part above in a separate function and call it also from OnRemoteVoiceInfo(..) to make sure the IAudioOut is acually initialized before Start(..) is called on it.
    So it looks like this now -
    protected override void Awake()
            {
                base.Awake();
                var factory = InitAudioOut();
    
                #if !UNITY_EDITOR && UNITY_PS4
                this.audioOutput = new Photon.Voice.PS4.PS4AudioOut(PS4UserID, factory);
                #else
                this.audioOutput = factory;
                #endif
            }
    
      IAudioOut InitAudioOut()
            {
                var audioSource = this.GetComponent<AudioSource>();
                IAudioOut audioOut = new AudioStreamPlayer
                (
                    new VoiceLogger(this, "AudioStreamPlayer", this.LogLevel), 
                    new UnityAudioOut(audioSource), "PhotonVoiceSpeaker:", this.Logger.IsInfoEnabled
                );
    
                return audioOut;
            }
    
      internal void OnRemoteVoiceInfo(RemoteVoiceLink stream)
            {
                this.remoteVoiceLink = stream;
                this.remoteVoiceLink.FloatFrameDecoded += this.OnAudioFrame;
                this.remoteVoiceLink.RemoteVoiceRemoved += this.OnRemoteVoiceRemove;
                VoiceInfo voiceInfo = stream.Info;
    
                if (this.audioOutput == null)
                    this.audioOutput = InitAudioOut();
    
                this.audioOutput.Start(voiceInfo.SamplingRate, voiceInfo.Channels, voiceInfo.FrameDurationSamples, this.PlayDelayMs);
            }
    Can anyone reproduce this issue?

    Thank you and Kind Regards !
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @MartinD,

    Thank you for choosing Photon!

    Speaker has been updated in 2.3 to address a NullReferenceException (audioOutput field null as not yet initialized in Awake) related to the timing of OnRemoteVoiceInfo and Awake. Starting from 2.3, we support cases where OnRemoteVoiceInfo is called before Awake.

    It's now like this:

    protected override void Awake()
            {
                base.Awake();
                Func<IAudioOut> factory = () => new AudioStreamPlayer(new VoiceLogger(this, "AudioStreamPlayer", this.LogLevel),  
                    new UnityAudioOut(this.GetComponent<AudioSource>()), "PhotonVoiceSpeaker:", this.Logger.IsInfoEnabled);
    
                #if !UNITY_EDITOR && UNITY_PS4
                this.audioOutput = new Photon.Voice.PS4.PS4AudioOut(PS4UserID, factory);
                #else
                this.audioOutput = factory();
                #endif
                this.StartPlaying();
            }
    
            internal void OnRemoteVoiceInfo(RemoteVoiceLink stream)
            {
                if (stream == null)
                {
                    if (this.Logger.IsErrorEnabled)
                    {
                        this.Logger.LogError("RemoteVoiceLink is null, cancelled linking");
                    }
                    return;
                }
                if (this.IsLinked)
                {
                    if (this.Logger.IsWarningEnabled)
                    {
                        this.Logger.LogWarning("Speaker already linked to {0}/{1}, cancelled linking to {2}/{3}",
                            this.remoteVoiceLink.PlayerId, this.remoteVoiceLink.VoiceId, stream.PlayerId, stream.PlayerId);
                    }
                    return;
                }
                this.remoteVoiceLink = stream;
                this.remoteVoiceLink.RemoteVoiceRemoved += this.OnRemoteVoiceRemove;
                this.StartPlaying();
            }
    
    internal void OnRemoteVoiceRemove()
            {
                bool wasStarted = this.started;
                if (this.audioOutput != null)
                {
                    this.audioOutput.Stop();
                    this.started = false;
                }
                this.Actor = null;
                if (this.OnRemoteVoiceRemoveAction != null) { this.OnRemoteVoiceRemoveAction(this); }
                if (this.remoteVoiceLink != null)
                {
                    this.remoteVoiceLink.RemoteVoiceRemoved -= this.OnRemoteVoiceRemove;
                    if (wasStarted)
                    {
                        this.remoteVoiceLink.FloatFrameDecoded -= this.OnAudioFrame;
                    }
                    this.remoteVoiceLink = null;
                }
            }
    
            internal void OnAudioFrame(float[] frame)
            {
                this.audioOutput.Push(frame);
            }
    
            private void Update()
            {
                this.audioOutput.Service();
            }
    
            private void StartPlaying()
            {
                if (!this.started && this.audioOutput != null && this.IsLinked)
                {
                    VoiceInfo voiceInfo = this.remoteVoiceLink.Info;
                    this.audioOutput.Start(voiceInfo.SamplingRate, voiceInfo.Channels, voiceInfo.FrameDurationSamples, this.PlayDelayMs);
                    this.remoteVoiceLink.FloatFrameDecoded += this.OnAudioFrame;
                    this.started = true;
                }
            }



    So first, update to 2.3 and then try again.

    But if any one player leaves and rejoines a room he can not be heared by others.
    As it turns out the IAudioOut on the the Speaker in question gets not properly initialized when rejoining a room.
    But I can't understand how this is related to your description of the issue. If a player joins and rejoins, the Speaker prefab either gets instantiated again or reused. Either ways, Awake and OnRemoteVoiceInfo will be called once at least and in order for the audio to be played, the Speaker must be enabled and attached to a GameObject active in hierarchy for the MonoBehaviour Update method to be called.
  • Hi! I'm facing the same or similar issues (using latest Photon Voice). There seems to be some racing issue perhaps.

    When one Player joins a room (in my app Chatty), they often cannot be heard, even though they can hear others (so the Photon Voice room must be ok). The issue can be fixed if they turn off the microphone, then turn it on again (voiceRecorder.StopRecording() -> voiceRecorder.StartRecording()). So I'm trying to Invoke this reset 1 second after OnJoinedRoom as flaky workaround, even adding some time between StopRecording and StartRecording.

    However, even now, when then the *other* person leaves and later re-enters (note I always add some delay before switching back), the original person who had their mic turned off and on again then once more cannot be heard.

    My settings are "Auto Connect and Join" On, and "Auto Leave and Disconnect" Off (as having the latter On caused voice to not work, though that may have been the same issue).

    Help please?
  • Update: I'm now *also* resetting the Mic of those who were already in the room, 0.5s after someone else's joining person was instantiated. Which seems crazy enough, but it does seem to help (not sure how reliable).
  • Nope, that also won't fully solve it :/
  • Philipp
    Philipp
    edited November 2019
    I may have something of interest. Background of the test flow:

    1. When A and B joins in a room the first time, all works. (It doesn't matter who joined the room first.)
    2. Now B leaves to another room, then comes back to A's room, and suddenly cannot hear A anymore.
    3. When A however toggles off and on their Recorder IsRecording, B can again hear A. (If there's a C in the room, they were able to hear A all along though, so A's recording is fine.)

    Here's the interesting bit: On B's client, VoiceConnection OnRemoteVoiceInfo(...) is never called again when they return to A's room! When A turns off and on their IsRecording, though, it forces OnRemoteVoiceInfo to be re-sent, fixing the voiceLink. This suggests to me that maybe the voiceLink is not re-established (I understand there's some voiceLink caching going on, not sure if related; whatever it is, there is a difference between the working first meet and non-working second meet of B and A).

    For reference, the whole Pun-room auto-Follow of the Voice Network works fine, I've checked all the Voice room values (though naturally, the voice room will take a second or so to catch up with the Pun room).

    (I'm happy to share my full project source code privately if needed, if it helps shining light on this.)
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @Philipp,

    Thank you for all the details and the repro steps.
    I'm going to work on this now and post an update here.
  • Thank you!
  • JohnTube
    JohnTube ✭✭✭✭✭
    Posting and update here since @Philipp confirmed the fix is working:

    This will be part of the next Photon Voice update, 2.12.

    In current version 2.11, in file: "Assets\Photon\PhotonVoice\PhotonVoiceApi\LoadBalancingTransport.cs"

    Change onStateChangeVoiceClient method, lines 280-295, from:
           void onStateChangeVoiceClient(ClientState fromState, ClientState state)
            {
                switch (state)
                {
                    case ClientState.Joined:
                        this.voiceClient.onJoinChannel(VOICE_CHANNEL);
                        if (this.voiceClient.GlobalInterestGroup != 0)
                        {
                            this.LoadBalancingPeer.OpChangeGroups(new byte[0], new byte[] { this.voiceClient.GlobalInterestGroup });
                        }
                        break;
                    case ClientState.Disconnected:
                        this.voiceClient.onLeaveChannel(VOICE_CHANNEL);
                        break;
                }
            } 
    to:
    void onStateChangeVoiceClient(ClientState fromState, ClientState state)
            {
                if (fromState == ClientState.Joined)
                {
                    this.voiceClient.onLeaveChannel(VOICE_CHANNEL);
                } 
                else if (state == ClientState.Joined)
                {
                    this.voiceClient.onJoinChannel(VOICE_CHANNEL);
                    if (this.voiceClient.GlobalInterestGroup != 0)
                    {
                        this.LoadBalancingPeer.OpChangeGroups(new byte[0], new byte[] { this.voiceClient.GlobalInterestGroup });
                    }
                }
            }   
  • Excellent, thanks!