Current Buffer Lag - reduce faster in Speaker
in Photon Voice
In the Speaker component using PUN for Voice (v2), there's a display for "Current Buffer Lag". It seems to reflect the current lag from speaking to hearing in 10ths of ms (it usually hovers around 8500 and is about 0.85s of lag).
Sometimes for whatever reason (dropped signal, pausing, resizing window etc) the buffer lag increases, sometimes getting as high as 30000 - 3 full seconds lag. When it does this, it automatically "chipmunks" the voice to play it back faster, and brings the buffer lag back to normal, around 8500.
My Q is, how can I control this? It happens very slowly, with chipmunking often lasting 10-15 seconds for a few seconds of lag. I'd like to make the lag accommodation go faster, or - ideally - control the samples themselves. For example, skipping every other sample to make the voice talk very fast, rather than just compressing samples together and raising the pitch.
How can I modify/access that? Thanks!
Image shows the component with buffer lag quite high, at 2.6secs:
Sometimes for whatever reason (dropped signal, pausing, resizing window etc) the buffer lag increases, sometimes getting as high as 30000 - 3 full seconds lag. When it does this, it automatically "chipmunks" the voice to play it back faster, and brings the buffer lag back to normal, around 8500.
My Q is, how can I control this? It happens very slowly, with chipmunking often lasting 10-15 seconds for a few seconds of lag. I'd like to make the lag accommodation go faster, or - ideally - control the samples themselves. For example, skipping every other sample to make the voice talk very fast, rather than just compressing samples together and raising the pitch.
How can I modify/access that? Thanks!
Image shows the component with buffer lag quite high, at 2.6secs:
0
Answers
-
Hi @nickyonge,
Thank you for choosing Photon!
We have added two extra playback delay settings.
Please update to >2.22 and see if it helps out of the box otherwise tweak those values.
you can find more info here.
if the issue persists do let us know.
1 -
Tried updating and while I see the new playback delay settings, they don't seem to resolve what I'm looking for. I can adjust when it seems to kick in, but if delays build up enough that the "chipmunking" is noticeable, it can still take a long time for it to slowly return to a normal pitch. Please lmk if there's some other way to access that buffer adjustment
0 -
Hi @nickyonge,
What delay values did you try?
I think you should start by setting MaxDelayHard <= MaxDelaySoft and for the MinDelaySoft try to increase by a 100ms step or double it every time (starting from default 200) while keeping MaxDelaySoft = 2 * MinDelaySoft, repeat until you find a good combination that gives you the intended results.0 -
Currently trying 50 / 200 / 200 for MinSoft / MaxSoft / MaxHard. Tried a handful of different things like 50 / 100 / 400, 100 / 250 / 1000, a bunch. There's often some kind of popping noise too. It takes a bit of time to test but we're trying different settings.
Can you explain in a bit more detail what these do? What is soft vs. hard, and for what use cases would you want to change those values?0 -
Hi @nickyonge,
As I have tried to explain here already, here is more info / context:
Advanced playback delay settings were introduced to fix a previous issue.
Those changes were made in the audio player (UnityAudioOut) to better handle network delay variations.
Playback delay settings:
- MinDelaySoft: the lowest bound of target delay: the audio player tries to keep the delay above this value.
- MaxDelaySoft: upper bound of target delay: audio player tries to keep the delay below this value.
- MaxDelayHard: audio player guarantees that the delay never exceeds this value.
Playback behaviour:
a. While the delay is between 1 and 2, nothing happens.
b. if it's between 2 and 3, data is resampled to catch up with the input stream.
c. When the delay > 3, it's just set to target value (the average of 1 and 2, default is (200 + 400) / 2 = 300).
Conditions:
- 0 <= MinDelaySoft
- MaxDelaySoft > MinDelaySoft
- if MaxDelayHard <= MaxDelaySoft: in this case no resampling will happen.
We will add all info to the documentation soon.0 -
Thank you for the reply!
For "playback behaviour", b: "data is resampled to catch up with the input stream". How and where is it resampled? I feel like this is the parameter I'd most like to play with, to make the resampling either more or less aggressive, or to just modify the algorithm to suit our needs.0 -
Hi @nickyonge,
The idea is to tweak the delay values until you find a good compromise.
It's all done in UnityAudioOut, mainly Service and Flush.
You could also make your own IAudioOut implementation for playback and use it in Speaker.
You could try uncommenting "//#define USE_ONAUDIOFILTERREAD" and see if it helps, though this playback method that makes use of Unity's built-in audio pipeline via OnAudioFilterRead callback will not work w/ spatial audio (3D positioning).0
