Photon microphone (no sound) vs Unity microphone (sound)

@JohnTube Using Photon Voice 2.1.1 on Unity 2018.2.18f1 on an MSI Dominator Laptop with Realtek High Definition Audio driver. Microphones include built in mic, HTC Vive.

Running the DemoVoiceUI demo, the Unity microphone successfully detects sound, while the Photon microphone does not.

Condensed log below from DemoVoiceUI where Unity mic is active first, then switch to Photon mic:

Load scene 'Temp/__Backupscenes/0.backup' time: 1.081285 ms
ConnectAndJoin.ConnectNow() will now call: VoiceConnection.ConnectUsingSettings().
...
Photon.Voice.Unity.UtilityScripts.ConnectAndJoin:ConnectNow() (at Assets\Photon\PhotonVoice\Code\UtilityScripts\ConnectAndJoin.cs:66)
Photon.Voice.Unity.UtilityScripts.ConnectAndJoin:OnEnable() (at Assets\Photon\PhotonVoice\Code\UtilityScripts\ConnectAndJoin.cs:55)

(Filename: Assets/Photon/PhotonVoice/Code/UtilityScripts/ConnectAndJoin.cs Line: 66)

Refresh Mics
...
(Filename: Assets/Photon/PhotonVoice/Demos/DemoVoiceUI/MicrophoneDropdownFiller.cs Line: 121)

[PV] Remote voices cleared
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[Scripts.Recorder] Type Convertion set to Short. Audio samples will be converted if source samples type differs.
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[Scripts.Recorder] Setting recorder's source to Unity microphone device Microphone (Realtek High Definition Audio)
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[Scripts.Recorder] [PV] MicWrapper: initializing microphone 'Microphone (Realtek High Definition Audio)', suggested
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[Scripts.Recorder] [PV] MicWrapper: microphone 'Microphone (Realtek High Definition Audio)' initialized, frequency = 48000, channels = 1.
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

PUNVoice: Creating local voice with source samples type conversion from float to short.
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[PV] Local v#1 ch#1 Sending info: c=AudioOpus f=48000 ch=1 d=10000 s=480 b=30000 w=0 h=0 ud= ev=0
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[PV] Local v#1 ch#1 added enc: c=AudioOpus f=48000 ch=1 d=10000 s=480 b=30000 w=0 h=0 ud=
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[WebRTCAudioProcessor] WebRTCAudioProcessor create sampling rate 48000, frame samples 480
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[PV] Local v#1 ch#1: Starting data encode thread
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[Scripts.Recorder] Recorder.MicrophoneType changed, Recorder requires reinitialization for this to take effect.
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[PV] Local v#1 ch#1 remove sent
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[PV] Local v#1 ch#1: Exiting data encode thread
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[PV] Local v#1 ch#1 removed
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[Scripts.Recorder] Setting recorder's source to Photon microphone device 0
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[Scripts.Recorder] Setting recorder's source to WindowsAudioInPusher
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

[PV] Local voice #2 audio source frequency 16000 and encoder sampling rate 48000 do not match. Resampling will occur ...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1830)

[PV] Local v#2 ch#1 Sending info: c=AudioOpus f=48000 ch=1 d=10000 s=480 b=30000 w=0 h=0 ud= ev=0
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[PV] Local v#2 ch#1 added enc: c=AudioOpus f=48000 ch=1 d=10000 s=480 b=30000 w=0 h=0 ud=
...
(Filename: Assets/Photon/PhotonRealtime/Code/LoadBalancingClient.cs Line: 1834)

[WebRTCAudioProcessor] WebRTCAudioProcessor create sampling rate 48000, frame samples 480
...
(Filename: Assets/Photon/PhotonVoice/Code/VoiceLogger.cs Line: 92)

Comments

  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @john__primitive,

    I appreciate your efforts and thanks for all your reports and help :)

    Just to make sure:

    - This happens on both Unity Editor and on a build? build platform? is it "HTC Vive"?
    - The logs are from Unity Editor I assume, and nothing useful after "[WebRTCAudioProcessor] WebRTCAudioProcessor create sampling rate 48000, frame samples 480"
    - how do you test this? using DebugEchoMode or from two clients?
    - did you try disabling or fiddling with VoiceAudioPreprocessor component? also deactivating Voice Activity Detection everywhere?
    - what speakers do you use and expect sound from?
  • @JohnTube
    1 - yes, for this particular laptop the Photon mic is inactive in the editor and on a build and it's the same issue for all mics that the laptop uses
    2 - yes, logs are from the editor and the last log is the last one that the Photon logger outputs
    3 - I'm watching the level meter on the recorder in the scene for activity (which does not appear at all when the Photon mic is selected). The debug audio is silent as well, and running the scene on two separate clients also reveals that the laptop with Photon mic is inactive
    4 - VAD is deactivated
    5 - the speakers on the laptop work fine

    I hope this all helps. This very well may be an issue with this specific model of laptop (MSI Dominator GT 72VR) since the audio drivers are not great, but it's just weird that the Unity mic works and the Photon mic does not. Again, if there is a good way to switch between the two in scripting when one isn't working that would be a great temporary solution.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @john__primitive,

    Does the Unity Editor show the same number and names of devices for both Unity and Photon microphone types?
    Could you try Photon Voice 2.1.2? we didn't change much but...
    This very well may be an issue with this specific model of laptop (MSI Dominator GT 72VR) since the audio drivers are not great
    So on other machines it works fine?
    it's just weird that the Unity mic works and the Photon mic does not
    yes
    Again, if there is a good way to switch between the two in scripting when one isn't working that would be a great temporary solution.
    There is no good way to detect that one microphone input source is not working. What I can think of as a workaround if you really need this is to have a timer, with both recorder VAD and VoiceAudioPreprocessor altogether disabled, that starts when you enable transmission on the recorder and checks after some time that Recorder.IsCurrentlyTransmitting is true.
  • @JohnTube I'm following up on this thread because this issue is still persisting in Photon Voice 2.5. This is on a new setup (on a friend's PC). I'll see if I can get info about the number and names of microphone devices in the editor to see if they are different.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @john__primitive,

    You did not answer question from my previous comment.
    Is this issue only reproducible on a single machine: laptop MSI Dominator GT 72VR.

    If this issue happens on a single machine (MSI Dominator GT 72VR) maybe the issue is with the hardware or the driver on the audio devices?

    Maybe my colleague @vadim may know better how to debug and investigate issues of Photon standalone Unity native plugin on that machine.
  • beppert
    beppert
    edited April 2019
    Hello. Same project, same issue. The Photon mic does not work but the Unity mic does. I'm seeing levels reported in Unity for the Photon mic but no voice is being transmitted. When I switch to using Unity microphone type voice is being sent.

    PC info: ASRock motherboard, Intel i7-4790K CPU, Oculus Rift microphone & speakers.

    Here are logs from 2 different runs. The Photon Microphone Type does not show any initialization info but the Unity Microphone Type does.

    Photon Mic:
    https://www.dropbox.com/s/crmd4iomcdbm3zc/Photon Mic.PNG?dl=0

    Unity Mic:
    https://www.dropbox.com/s/vs4w6x860yx00ax/Unity Mic.PNG?dl=0
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited April 2019
    Hi @beppert,

    Thank you for choosing Photon!

    can we get rid of the speaker not linked warnings first?
    when you test this, don't rely on the level meter or any other indicators that you may think are showing that voice is being properly sent.
    to test this simply make sure you hear voice being transmitted either on the same client using DebugEchoMode or on a remote client.

    Oculus Rift microphone & speakers
    According to one of our customers, when
    running a VR game and when you start one of those, it will activate the goggles and automatically switch to the respective audio device.
    To handle this, we have lately added "Recorder.ReactOnSystemChanges" could you enable that from the inspector or from code before transmission?

    The Photon Microphone Type does not show any initialization info
    You can add logs inside WindowsAudioIn
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hey @john__primitive, @beppert,

    Any updates on this?

    Do you set the Recoder.PhotonMicrophoneDeviceId from Unity Editor or from code?
    Maybe you should try keeping its default value -1.
  • I'm seeing a few problems:
    1. Unity mic is still causing crashing. I believe I worked around this before by using the Photon mic
    2. When I use the Photon mic, the Microphone device chosen doesn't get saved.
    3. The Photon mic is always defaulting to a microphone that isn't the "default" Windows microphone.
    (support, yes I did also send you an email about these same issues...so you don't need to answer me twice).
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @BuilderWorldVR,

    1. How can we reproduce the crashes using Unity mic, we want to fix this: minimal repro steps, Photon Voice version, Unity version, platforms, does it happen in Editor only or also on builds or builds only?
    2. Yes, that is expected. see my answer here
    3. hmm, this is the first time we get such report. we need to check and see if it's always the case and if it's possible to change or if it's worth changing.