Weird audio distortion in Photon

Hey all,

We've been having some odd problems with photon voice since updating to the latest version from an older one. In some sessions, audio will have an odd 'stuttering' quality, like half the frames are getting dropped. I'm not talking about an occasional hitch, which is to be expected given network weather, but bad enough that the speaker can become unintelligible. It also happens over LAN, where bandwidth shouldn't be an issue, and can often be fixed by restarting the client.

Has anyone experienced anything like this? Any idea what might be up? We're using the HTC Vive microphone for audio capture, and it doesn't seem to be a hardware fault (the audio does not show up when recording audio directly).

Thanks for any help you can give!
-Andre

Comments

  • Oh, I'm not sure if it's relevant, but we're also getting a *ton* of "PUNVoice: PhotonVoiceSpeaker: player underrun:" errors in the log.
  • Sorry, that should be "audio issue does not show up" in the OP.
  • Hi,
    Does this happen from the very beginning of the session or may appear after some time of good quality transmission?
    When 2 clients listen to the same 3rd client, is it possible that 1st client playbacks normally while 2nd produces distorted sound? This can help understand on which side the problem is.
    What microphone sampling rate and opus sampling rate do you use?
    Can you reproduce the issue with the TestVoice demo from Photon Voice package imported in new empty project with default settings?
    Please send us logs from both transmitting and receiving clients.
    The problem may be related to playback jitter buffer. You can try to increase in "Play Delay Ms" property of Photon Voice Settings component.
  • AndreFOO
    AndreFOO
    edited June 2017
    Thanks for the help! To answer your questions:

    * It seems to occur a soon as the client logs in, and may worsen over time.

    * We've definitely had a situation where two clients can hear clearly, but the third client hears all network audio as distorted.

    * We use a 48k sampling rate, a 20 ms frame delay, and a bitrate of 50,000. The play delay ms is currently set to 150, although I've tried tuning it higher to resolve the issue,with no luck.


    I will try to import a clean instance of photon and photon voice and see if I can repro the issue.
  • Update: not getting the issue in a clean project. Going to try reinstalling photon and photon voice and see if that resolves the issue.
  • AndreFOO
    AndreFOO
    edited June 2017
    So I reinstalled / upgraded both photon and photon voice, and am still seeing the issue. I got some logs for you!

    Here's the relevant section of the log from the client experiencing the issue. It goes on like this for the whole run. not sure if that helps at all.

    Initializing setup!

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: [PV] Frame event for voice #1 of not inited channel 1

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: [PV] Frame event for voice #1 of not inited channel 1

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: [PV] Frame event for voice #1 of not inited channel 1

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: Player 1 voice #1 speaker created.

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 13312/21600(19650) = 8288

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 45888/54240(53028) = 8352

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 65216/73440(71701) = 8224

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 88128/96480(95145) = 8352

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 131904/140160(138693) = 8256

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 149600/157920(156111) = 8320

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 190656/199200(197738) = 8544
  • And here's the relevant log from the client that was transmitting the voice (and not experiencing the issue)

    Initializing setup!

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    GfxDevice: creating device client; threaded=1
    GfxDevice: creating device client; threaded=1
    Initializing setup!

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: Player 2 voice #1 speaker created.

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player overrun: 2535040/2535360(2535360) = 320

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: PhotonVoiceSpeaker: player underrun: 2536000/2544480(2540774) = 8480

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: Player 2 voice # 1 speaker unlinked.

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    PUNVoice: Voice #1 of player 2 not found.

    (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 42)

    Setting up 1 worker threads for Enlighten.
    Thread -> id: 29a4 -> priority: 1
    OpenVR Shutdown

    (Filename: Line: 701)
  • Can provide an audio clip of what the voice sounds like, if that would help.
  • You distortion may be like http://forum.photonengine.com/discussion/comment/34950/#Comment_34950
    If not, please send a clip.
    Also send full logs (from the very start) from both clients built with Pun Logging set to 'Full' in Photon Server Settings.
  • AndreFOO
    AndreFOO
    edited June 2017
    We also noticed today that it seems to be related to which audio interface is in use by the client that's hearing the distortion. For streaming high quality audio over photon we sometimes use the focusrite scarlett 2i2, and after some testing, it seems like that's what's causing the audio bug.

    Ordinarily, I'd say let's just work around the issue, but there are some external reasons we need to use that hardware in particular, so it'd be really helpful if we could get it working.
  • We also tested with both 24k and 48k sampling rates, and got the same behavior on both.
  • Sample sounds "slightly" distorted but is still distinguishable (the sample I posted link to earlier is completely unrecognizable).
    You loose a lot of packets according to your logs. Opus can recover if losses are not too high. Maybe network quality is a bit lower than opus can handle w/o audible distortions.
    Please try 'reliable' mode which delivers all encoded packets to decoder. Note that in this case delay may be higher and "Play Delay Ms" may need to be increased.
    in PhotonVoiceRecorder.cs, find 2 occurrences of this.voice = PhotonVoiceNetwork.CreateLocalVoice(...); and add line this.voice.Reliable = true; right after voice creation.
  • AndreFOO
    AndreFOO
    edited August 2017
    Sorry for the late reply, had to work through some crunch.

    The audio I posted is towards the lower end of how garbled it gets - it definitely gets bad enough that speech is not intelligible (although never as bad as the example you gave). But we're using this in an application where the audio passing over the network is going to be recorded for a large number of viewers in a professional setting (think a podcast). Even mild distortion is obviously not acceptable for our use case, since it's constant and consistent.

    It's also definitely not just network weather- this happens both over the internet, and over hard-wired LAN. We also routinely run other voice chat applications over the same networks, without seeing similar problems. If there's a high rate of packet loss, it's gotta be happening software side.

    I gave teliable mode a try (per your instructions), but it did not resolve the bug, even with a 1000 ms play delay.

    Any other things we might try to narrow down the problem?
  • Hello!

    Unfortunately we are also experiencing the same audio stuttering problem.
    In our case we are using the OVRPlugin in our project.
    PUNVoice is working fine for two players but when more than two players are talking together in a room we get the "PUNVoice: PhotonVoiceSpeaker: player underrun ..". warnings.
    I tried every proposed solution above but whitout any success so far...

    Thanks in advance!
  • vadim
    vadim mod
    edited October 2018
    Hi,

    Player audio streams are separated clearly. Not sure how they can interfere. Is connection bandwidth enough to receive streams from several players? Do you experience the same problems when running demos from PhotonVoice package?
    Please share the logs from at least 1st player in the room. Set Photon network logging level to Informational in Photon settings.

    About hardware specific bugs. We rely on Unity microphone api. If app works for one microphone but does not for other, this is most likely Unity bug. It may be fixed with one of Unity updates. For PC, Mac, iOS and Android, it's possible to switch to Photon native audio capture plugin. Make sure that mic type in Photon Voice settings set to Unity (default) when testing mentioned platforms.