Using PUN Voice with OVR Lip Sync

NotYourSawtoothWave
2018-12-02 20:57:27

Hi... I've been using the Oculus Lip Sync plugin (OVRLipSync) with PUN Voice for a while and it has worked very nicely with just one limitation: each client cannot see their own lip syncing (in mirrors, preview render targets, etc). This is because the OVRLipSyncContext script uses a reference to an AudioSource to perform its lip sync calculations on.

So my avatar prefab has an AudioSource, a PhotonVoiceSpeaker and PhotonVoiceRecorder as well as OVRLipSyncContext. When its a remote player, PhotonVoiceSpeaker is playing through the AudioSource which the OVRLipSyncContext references. When its a local player, the AudioSource isn't producing any sound.

I don't want the local player to hear their own voice, just want for their lip syncing to work. Can anyone recommend a way to achieve this?

Comments

vadim
2018-12-03 08:48:16

Hi

How do you do this without PUN Voice? Add AudioSource, mute it, send audio to it? Whatever works w/o PUN Voice, should work with. You can try to use existing AudioSource or add new.
To get copy if mic input, you need to implement Voice.IProcessor and add it to audio processing pipeline. You can find a sample in DelayProcessor.cs from PhotonVoice1 package. It should be similar in PhotonVoice2.

JohnTube
2018-12-03 09:24:01

Hi @NotYourSawtoothWave,

Thank you for choosing Photon!

I see you are using Photon Voice 1 and probably by "PUN Voice" that you use it in the normal use case with PUN1 integration.

To make OVRLipSync work for the local player also, could you try enabling DebugEchoMode in PhotonVoiceRecorder and mute local player's AudioSource? This means the local player will receive his voice from the server also (mind the bandwidth!).

CompuGeniusPrograms
2020-05-10 20:42:45

@JohnTube wrote: »

Hi @NotYourSawtoothWave,

Thank you for choosing Photon!

I see you are using Photon Voice 1 and probably by "PUN Voice" that you use it in the normal use case with PUN1 integration.

To make OVRLipSync work for the local player also, could you try enabling DebugEchoMode in PhotonVoiceRecorder and mute local player's AudioSource? This means the local player will receive his voice from the server also (mind the bandwidth!).

I'm not exactly sure what you mean, but I think I did it correctly, yet it's not working. Check out my thread here: https://forum.photonengine.com/discussion/16098/lip-sync-w-oculus-avatar

JohnTube
2020-05-14 16:46:41

closing, see here