Calibration of VAD not stopping

Hello,
I'm trying to set up Recorder VAD. It seems that recorder initialization is successful but when I start calibration process it does not seem to end. I tried starting it both via script and with the button in the inspector.
I tried setting Log Level to ALL and press calibrate button in inspector but I did not get any messages. After pressing the button inspector UI changes from "Calibration time(ms) 200" to "calibrating 200 ms" and the number is not going down.

Anyone has any idea what might be happening here?

I am using Photon Voice 2.8 and Unity 2019.2.0f1.

Comments

  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @Gom,

    Thank you for choosing Photon!

    Does this happen all the time?
    Does this happen with demo scenes?
    How can we reproduce?

    Does Recorder.VoiceDetectorCalibrating return true all the time?
  • Yes, it was returning true all the time. Now playing around in the demo scene I noticed that calibration does not end if Transmit Enabled set to false.
    If Transmit Enabled is set to false I can still see the Calibrate button but after pressing it calibration does not end and VoiceDetectorCalibrating always returns true.
    Calibrate button only disappears if IsRecording is set to false.

    I set both IsRecording and Transmit Enabled to true and it seems to be working now.

    Now I am confused what exactly Transmit Enabled does. As I understood VAD was supposed to toggle it?

    Thanks
  • The only place I found describing this is here: https://doc.photonengine.com/en-us/voice/v2/getting-started/recorder. But I can not figure out what all of the booleans do there. Is there somewhere a more detailed description of this component?
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @Gom,

    We have reproduced this.
    We will fix this and let you know.
    We will either share quick fix here or release an update.
  • Thanks, will be waiting for an update
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited August 2019
    Hi @Gom,

    I discussed internally with my colleague @vadim who works on the "low-level" PhotonVoiceApi (Photon Voice Core or C# API).

    To save CPU, the entire audio processing pipeline disabled when TransmitEnabled is set to false, not transmission only.
    Stopping transmission during calibration is not a common use case and not recommended, it's an edge case.

    Making calibration stops when a transmission is disabled will require a big refactoring and redesign of the processing pipeline. We think it's not worth it for now.
    So we will make it look like calibration is paused then resumed if the transmission is toggled in between.

    For now, from the Unity high-level layer, I changed two things in the Recorder component:

    Recorder.VoiceDetectorCalibrating will return false if Recorder.TransmitEnabled is false:
    public bool VoiceDetectorCalibrating { get { return voiceAudio != null && this.TransmitEnabled && voiceAudio.VoiceDetectorCalibrating; } }
    Recorder.VoiceDetectorCalibrate cannot be started if Recorder.TransmitEnabled is false:
    public void VoiceDetectorCalibrate(int durationMs, Action<float> detectionEndedCallback = null)
            {
                if (this.voiceAudio != null)
                {
                    if (!this.TransmitEnabled)
                    {
                        if (this.Logger.IsWarningEnabled)
                        {
                            this.Logger.LogWarning("Cannot start voice detection calibration when transmission is not enabled");
                        }
                        return;
                    }
                    this.voiceAudio.VoiceDetectorCalibrate(durationMs, newThreshold =>
                    {
                        this.GetThresholdFromDetector();
                        if (detectionEndedCallback != null)
                        {
                            detectionEndedCallback(this.voiceDetectionThreshold);
                        }
                    });
                }
            }
  • JohnTube
    JohnTube ✭✭✭✭✭
    Also, in the Editor we hide the Calibrate button when TransmitEnabled is false, "RecorderEditor.cs":

    before:
                        if (recorder.IsRecording)
                        {
                            if (GUILayout.Button("Calibrate"))
                            {
                                recorder.VoiceDetectorCalibrate(calibrationTime);
                            }
                        }
    after:
                        if (recorder.IsRecording && recorder.TransmitEnabled)
                        {
                            if (GUILayout.Button("Calibrate"))
                            {
                                recorder.VoiceDetectorCalibrate(calibrationTime);
                            }
                        }
  • Thank you for help!

    For now I am using Recorder to only detect voice activity to get the speech and send it to our speech processing to transform it to text, try to understand it and give the answer with AI. So I don't think we need transmition, just recording? Though we will be doing voice chat later too. Just figured that as Photon Voice already has VAD and we will be using it later so there's no point in doing our own VAD too.
    It is working well now, thanks