Is it possible to stream audio?

shadowsora
2018-02-20 17:50:16

Hi all,

I'm working on a multiplayer music-based game that will allow the master client to load their own sound files. Because of this, I can't tell the other clients to start playing these audio clips, so I was wondering if it would be possible to stream the audio from the master client to all other clients? It is also important that all clients hear the same audio, which is why I thought the streaming approach would be the best.

Thanks!

Comments

Kaiserludi
2018-02-21 15:08:11

Hi @shadowsora.

That sounds like a perfect match for Photon Voice, which sits on top of PUN and handles audio streaming for you. It doesn't really matter if the audio stream that you send through it contains voice data from the microphone or if you just play for example a prerecorded .wav file.

vadim
2018-02-21 15:41:29

HI,

For audio streaming you can use PhotonVoice package.
It has PUN inside, so you need to remove PUN from your project and add PhotonVoice.
PhotonVoice is designed for streaming captured microphone input in the first place but you can adapt it for any audio source. Try demos working with microphone first. You will find PhotonVoiceRecorder component which reads and broadcasts audio.
To change microphone to custom audio source, create class reading your audio source and implementing ExitGames.Client.Photon.Voice.IAudioReader interface. Look at AudioClipWrapper which streams audio clip assigned in editor. It does very similar to want you need.
Set PhotonVoiceRecorder's Source to Factory in editor and tell PhotonVoice to use factory creating instance of your class somewhere during app initialization (before PhotonVoiceRecorder created):
PhotonVoiceNetwork.AudioSourceFactory = (rec) => new MyAudioSource();
As long as client is connected to a voice room and PhotonVoiceRecorder is transmitting, Read(float[] buffer) method will be called on MyAudioSource instance. Calls frequency and buffer size are adjusted to meet sampling rate returned by SamplingRate property of MyAudioSource.

shadowsora
2018-02-23 13:10:07

Hi @Kaiserludi and @vadim. Thank you both for your help! I haven't had a chance to try this out yet but it sounds like Photon voice is just what I need for my game!

shadowsora
2018-03-25 16:42:28

@vadim wrote:

HI,

For audio streaming you can use PhotonVoice package.
It has PUN inside, so you need to remove PUN from your project and add PhotonVoice.
PhotonVoice is designed for streaming captured microphone input in the first place but you can adapt it for any audio source. Try demos working with microphone first. You will find PhotonVoiceRecorder component which reads and broadcasts audio.
To change microphone to custom audio source, create class reading your audio source and implementing ExitGames.Client.Photon.Voice.IAudioReader interface. Look at AudioClipWrapper which streams audio clip assigned in editor. It does very similar to want you need.
Set PhotonVoiceRecorder's Source to Factory in editor and tell PhotonVoice to use factory creating instance of your class somewhere during app initialization (before PhotonVoiceRecorder created):
PhotonVoiceNetwork.AudioSourceFactory = (rec) => new MyAudioSource();
As long as client is connected to a voice room and PhotonVoiceRecorder is transmitting, Read(float[] buffer) method will be called on MyAudioSource instance. Calls frequency and buffer size are adjusted to meet sampling rate returned by SamplingRate property of MyAudioSource.

Hi again,

I'm having some issues actually implementing this. Is there any example code?

Right now I am getting this error:
NullReferenceException: Object reference not set to an instance of an object
PhotonVoiceRecorder+c__Iterator0.MoveNext () (at Assets/Libraries/PUNVoice/Scripts/PhotonVoiceRecorder.cs:174)
UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)

Also is it possible to have two sets of recorders/listeners? I would like to have both voice chat and streamed audio in the game..

vadim
2018-03-28 13:04:42

At the line you mentioned, I see 'photonView.isMine' property accessed. That makes me think that PhotonVoiceRecorder is assigned to non-PhotonView object somehow.

PhotonVoiceRecorder is a part of integration of PhotonoVoice with PUN.
In case you removed PUN from your project, you need to use Voice client directly, w/o any helpers from PUNVoice folder.

You may have as many outgoing streams as you want. Call VoiceClient.CreateLocalVoice in Voice api or instantiate another PhotonVoiceRecorder object in PhotonVoice with PUN,

shadowsora
2018-04-02 13:40:21

Thanks! That was me being stupid.. didn't have a photon view component.

Edit 1: It seems to work now!
However the audio quality is not great, is there a way I can improve this?

Edit 2: It breaks up sometimes too...
So I am building up a buffer (say, readerBuffer) in my AudioReader, and when GetData(buffer[]) is called, I copy the first buffer.length elements from readerBuffer into buffer, and remove these from the readerBuffer. Is this the right approach?