Photon Mic versus Unity mic on different platforms and settings

PierreBlanc
2021-04-06 09:23:14

Hi there,

We are using Photon Voice v2.24 in a multi-platform collaborative app. Our product runs on PC/OSX VR and vrless as well as mobile IOS/Android.

Few questions:

1. On which platform is it recommended to use Photon or Unity mic ?

On IOS Photon mic seems to provides echo cancellation and improves the sounds quality. Should it be used on Windows too per instance ?

2. Issue with switching audio output

Using the Photon Voice 2 demo we managed to reproduce something we experienced in our app.
On a laptop with AirPods connected, when switching the output from the headset to the internal speaker, the audio (output and input) is lost until we reselect a mic again (select the same one or another one). This happens with both Unity and Photon mic.

3. WebRtcAudioDSP AGC

Using Photon Mic we experiences some low level issues, which we fixed by adding the WebRtcAudioDSP component, with AGC enabled.

4. WebRtcAudioDSP VAD

This component also provides VAD. Currently we use Photon VAD, which allows us to know when the voice is detected and display info about the state of the detection in our collaborative environment.

It seems that WebRtcAudioDSP is recommended instead of Photon VAD, but is it possible to plug something to know when the voice is detected with this method ?

5. WebRtcAudioDSP and AEC

We do not use this option. The question is that if i understand correctly on IOS/Android we shouldnt, as photon mic already provides built in hardware AEC. But on Windows, should we enable it ?

All the best !
Pierre

Comments

JohnTube
2021-04-06 10:51:42

Hi @PierreBlanc,

Thank you for choosing Photon and very very good questions!

In general we recommend Hardware processing using Photon microphone type (alone without WebRtcAudioDSP) for AEC / NS and AGC on supported platforms: Android, iOS, Windows and macOS.
WebRtcAudioDSP (software processing using WebRTC's Audio Processing Module) is recommended for VAD usage over default VAD and is recommended for the other AEC / NS / AGC in case the default behaviour with Photon microphone type is not enough or not good. WebRtcAudioDSP has its own unique features also, e.g. HighPass filter.

1. Always use Photon microphone type and have Unity as a fallback.

2. Interesting. We do not explicitly handle audio output unlike audio input that we focus on before. We still rely on Unity. Audio output selection or initialization may be related to the general audio setup though. What is the value of Recorder.ReactOnSystemChanges? Did you try enabling it and setting Recorder.SkipDeviceChangeChecks to true.

3. not a question :) but I can confirm that what you did is totally fine.

4. I will see with my colleague @vadim about this.

5. I think if you use Photon microphone type you shouldn't use WebRtcAudioDSP.AEC unless the results are not as good. Using both has the risk of the extra CPU overhead + audio alerted due to the double/over processing.

JohnTube
2021-04-06 11:11:01

hi @PierreBlanc,

4. Can you use Recorder.IsCurrentlyTransmitting to know when the voice is detected?

PierreBlanc
2021-04-07 13:26:23

Hi @JohnTube

Thanks for your answers.

Recorder.IsCurrentlyTransmitting : tested and working, thanks !

We need more testing on our side, we are using Quest/Quest2/Rift/RiftS/Vive, and we have audio issues on RiftS per instance, but i ll test with the demo to make sure its not coming from us.

1. ok so on IOS per instance, PhotonMic without WebRtcAudioDSP AGC results in way too low volume. Unity Mic volume was fine, but then we have echo. So we choose Photon mic on this platform (on all actually at the moment to test), and add the DSP plugin with AGC with default values.

How would you go about implementing a Fallback mechanism ? Do we just try to select Photon mic type, then a given mic from the enum, is there a callback we can listen to to understand it failed and so try with Unity Mic type and a given mic ?

2. this is happening as well in your DemoVoice as well

JohnTube
2021-04-07 13:40:57

Hi @PierreBlanc,

1. Microphone type fallback is already implemented and available out of the box. It's enabled by default and you could disable it via Recorder.UseMicrophoneTypeFallback.

2. I see. It's fine then to use WebRtcAudioDSP.AGC with Photon microphone type.