Output all voice conversations into .wav, .mp3 etc

KaiserBreath
2017-08-14 03:24:39

Hi,

I will be required to make a VR serious game that involves voice commands between the instructor and student as part of the training.

Will it be possible to record voice conversations using the plugin?

I needed to log them for review next time.

Thanks

Comments

vadim
2017-08-16 08:34:29

Hi,
Unity provides MonoBehaviour.OnAudioFilterRead which can be used to capture incoming audio per remote voice PhotonVoiceSpeaker (AudioSource) or record full application output mix if implemented on object with AudioListener.
For outgoing audio stream, you can create Voice.LocalVoiceAudioFloat.IProcessor and insert it in local voice processing pipeline. See DelayProcessor.cs

Towerss
2018-04-18 18:41:33

@vadim ,
Hi Guys,

I have a multiuser application which runs with Photon Unity Network (PUN) and Photon Audio. With Photon Audio, audio is streamed to the PUN server and then sent to all the players in the game. Photon Audio has a wrapper class on top of the core Unity audio classes to do their streaming. I would into implement in my application:

1. record to a wav file the incoming audio streaming from the Photon Audio server,
   and
2. redirect the Photon Audio streaming to another service feed.

I have a script to record and save the locally produced audio before it is streamed. I have read about the MonoBehaviour.OnAudioFilterRead, but I do not quite understand how could I use it to record the incoming audio stream.

Any guidance would be greatly appreciated.

vadim
2018-04-19 10:14:31

Hi,
You can add incoming streams handlers during app initialization like this:
PhotonVoiceNetwork.Client.OnAudioFrameAction += (playerId, voiceId, frame) => Debug.LogFormat("{0} {1} {2}", playerId, voiceId, frame[0]);
playerId and voiceId uniquely identify stream, frame is a float[] containing audio data.

Towerss
2018-04-20 13:50:01

@vadim wrote:

Hi,
You can add incoming streams handlers during app initialization like this:
PhotonVoiceNetwork.Client.OnAudioFrameAction += (playerId, voiceId, frame) => Debug.LogFormat("{0} {1} {2}", playerId, voiceId, frame[0]);
playerId and voiceId uniquely identify stream, frame is a float[] containing audio data.

Thank you @vadim , for your reply. Do you think would it be a sensible approach to tap into the file Photon Voice Speaker and use the parameter passed on OnVoiceLinked(int frequency, int channels, int frameSamplesPerChannel, int playDelayMs) and on OnAudioFrame(float[] frame) to create and save the wav file from there?

vadim
2018-04-23 07:56:06

Why PhotonVoiceNetwork.Client.OnAudioFrameAction does not work for you?
I forgot to mention PhotonVoiceNetwork.Client.OnRemoteVoiceInfoAction which provides info on new remote voice and PhotonVoiceNetwork.OnRemoteVoiceRemoveAction signalling remote voice removal.

Of course you can modify you copy of PUNVoice sources but it makes harder to update to new PUNVoice versions.

Towerss
2018-04-24 02:49:55

@vadim, Thank you so much. The update argument is quite on point. I will work with PhotonVoiceNetwork.Client.OnAudioFrameAction.

Towerss
2018-05-07 04:19:50

@vadim , I am having trouble figuring out how to use the float value returned by PhotonVoiceNetwork.Client.OnAudioFrameAction. Could you please lead me on how to store this stream? I think I need to store it before creating the wav file. Thanks in advance.

Towerss
2018-05-07 05:23:14

@vadim , If you don't mind having a look, this is what I have till now:

    public void SaveAudioStream()  
    {

        string filename = "TestWavFile";

        if (!filename.ToLower().EndsWith(".wav"))  
        {  
            filename += ".wav";  
        }

        //var filepath = filename;  
        var filepath = Path.Combine(Application.persistentDataPath, filename);

        Debug.Log("*** Filepath: " + filepath);


        // Make sure directory exists  
        Directory.CreateDirectory(Path.GetDirectoryName(filepath));


        FileStream writeStream = new FileStream(filepath, FileMode.Create, FileAccess.Write);  
        PhotonVoiceNetwork.Client.OnAudioFrameAction += (playerId, voiceId, frame) => Debug.LogFormat("***** {0} {1} {2}", playerId, voiceId, frame[0]);

        //I need to feed the MemoryStream buffer with the Audio data returned from PhotonVoiceNetwork.Client.OnAudioFrameAction  
        MemoryStream memstrm = new MemoryStream();

        int Length = 256;  
        Byte[] buffer = new Byte[Length];  
        int bytesRead = memstrm.Read(buffer, 0, Length);  
        // write the required bytes  
        while (bytesRead > 0)  
        {  
            writeStream.Write(buffer, 0, bytesRead);  
            bytesRead = memstrm.Read(buffer, 0, Length);  
        }

    }

vadim
2018-05-08 08:15:24

The code does not make sense. You set handler which simply logs on each frame. Not sure what code below does.
In case you want to capture entire stream, open file in OnRemoteVoiceInfoAction handler, write frame of audio data in OnAudioFrameAction handler and close the file in OnRemoteVoiceRemoveAction.
Or you can open and close file and update OnAudioFrameAction accordingly on user input.

Towerss
2018-07-25 11:08:44

Hi @vamin, It's me again. I would like to have your help with another recording related issue.

The first approach, I was using the OnAudioFrameAction handler to get all the frames received and save them in a wav file. I am using an "observer" instance that joins the session but only to record audio. That worked well for a single client connected apart from the "observer". As soon as I got two or more clients connected, the audio file was recording the float frames from the different clients overlapped, making the wav file useless.

The second approach, I was using MonoBehaviour.OnAudioFilterRead to get the frames on PhotonVoiceSpeaker.cs and the problem here is that I am having separate files for each client connected, not the whole conversation. The reason why I used this file is that my application says I already have an AudioListener somewhere, but I could not find it.

The third approach was using AudioStreamPlayer.cs without MonoBehaviour.OnAudioFilterRead. It worked the same way as in the second approach: there is a file opened for every client connected.

I would appreciate if you could give any other hint about how to record the whole incoming conversation in a single file.

Thank you.

Towerss
2018-07-26 03:39:39

Hi @vamin, no worries, all sorted now.

In the application I was debugging, there was a combination of [HideInInspector], HideFlags.HideInHierarchy, and HideFlags.HideInInspector applied to some gameobjects and components from third-party libraries.

I found the required gameobjects and components, and I used your suggested approach to use MonoBehaviour.OnAudioFilterRead with the gameobject that instantiates the AudioListener and everything is working now. I have one single WAV file with the whole conversation in good quality :)

Thank you.

RK_Rathod042
2021-09-05 05:19:40

@Towerss wrote: »

Hi @vamin, no worries, all sorted now.

In the application I was debugging, there was a combination of [HideInInspector], HideFlags.HideInHierarchy, and HideFlags.HideInInspector applied to some gameobjects and components from third-party libraries.

I found the required gameobjects and components, and I used your suggested approach to use MonoBehaviour.OnAudioFilterRead with the gameobject that instantiates the AudioListener and everything is working now. I have one single WAV file with the whole conversation in good quality :)

Thank you.

• hey, Are those methods you are mention above are working on v2 ?
• Can you please post your code ? because I don't found the "PhotonVoiceNetwork.Client.OnAudioFrameAction" class methods
• please help me-
• now I have completed saving outgoing audio but it's audio quilty is very bad-
• Check code here -> https://github.com/rk042/PhotonPunVoice/blob/main/Assets/Scenes/SaveOutVoice.cs

Towerss
2021-09-05 11:23:50

Hi @RK_Rathod042, I responded to your message. I do not have the whole code snipped, but this SO answer is the base for what I used: https://stackoverflow.com/questions/50302315/save-audio-stream-float-frames-as-wav-with-c-sharp

JohnTube
2021-09-05 22:08:26

Hi @Towerss, @RK_Rathod042,

Thank you for choosing Photon!

Photon Voice 2 contains two utility scripts to save incoming and outgoing audio streams to wav files.
Simply add "Assets\Photon\PhotonVoice\Code\UtilityScripts\SaveIncomingStreamToFile" to the same GameObject as VoiceConnection (or PhotonVoiceNetwork) or "Assets\Photon\PhotonVoice\Code\UtilityScripts\SaveOutgoingStreamToFile" to the same GameObject as Recorder.
Files are saved to Unity's persistent data folder.

RK_Rathod042
2021-09-15 11:37:01

@JohnTube thanks, there are working but the audio quality is very poor.

mecem1159
2021-12-01 15:45:21

Thanks for sharing.