Using FMOD with Photon Voice in Unity

I noticed that there were a FMOD folder under Assets/Photon/PhotonVoiceApi/Platforms, and tried to hook it up.

I have almost got it to work after these steps:

  1. Enable the define PHOTON_VOICE_FMOD_ENABLE
  2. Modify Speaker.cs to use a FMODUnity.StudioEventEmitter instead of an AudioSource (and Photon.Voice.FMOD.AudioOutEvent instead of Photon.Voice.Unity.UnityAudioOut).
  3. Make a looping programmer instrument event in FMOD (I made it 3 seconds long since that is how long buffer is created in AudioOut.cs)

So this is working and you can hear voice, but it keeps starting and stopping every 3 seconds, spamming this in the Console:

[PV] [FMOD] AudioOut (Event) EventCallback CREATE_PROGRAMMER_SOUND

[PV] [FMOD] AudioOut (Event) Sound Assigned to Event Parameter

[PV] [FMOD] AudioOut (Event) EventCallback SOUND_PLAYED

[PV] [FMOD] AudioOut (Event) EventCallback SOUND_STOPPED

[PV] [FMOD] AudioOut (Event) EventCallback DESTROY_PROGRAMMER_SOUND

[PV] [FMOD] AudioOut (Event) EventCallback CREATE_PROGRAMMER_SOUND

[PV] [FMOD] AudioOut (Event) Sound Assigned to Event Parameter

... (and so on) ...


Why is the sound stopped and recreated all the time?

Best Answers

  • Bexelius
    Bexelius
    Answer ✓

    None of my attempts have helped to remove the occacional garbled sound. Furthermore, I also found an issue where creating new speakers to occacionally failed to create an fmod sound.


    None of these issues happen when using UnityAudio, so we have decided to abandon this effort and revert to using the default UnityAudio output.

  • vadim
    vadim mod
    Answer ✓

    FMOD support in Photon Voice is experimental. Its main purpose is to demonstrate integration of 3rd party libraries with the Voice. Both FMOD.AudioOut and FMOD.AudioOutEvent work fine in my tests. But maybe my setup is different from what real FMOD users have. My experience in FMOD is limited. I just made it working in my simple test.

    Can you reproduce the issue with DemoVoiceUI or other demo from Photon Voice?

    For FMOD.AudioOutEvent, make sure that:

    • the event contains "Programmer Instrument"
    • Programmer Instrument is looped
    • the event is persistent
    • probably the event should be assigned to a bank
    • fmod project is specified in unity fmod settings
    • don't forget to run "build" in fmod studio after modifications


Answers

  • I solved it!


    If anyone attempts this themselves, you need to make the programmer instrument both async and looping, inside an equally long looping region.

  • JohnTube
    JohnTube ✭✭✭✭✭

    Hi @Bexelius,

    Thank you for choosing Photon and congratulations on making this work for you!

  • Glad to hear that FMOD works for you. Are your sure that FMODUnity.StudioEventEmitter is required? Switching to another AudioOut implementation should be enough. In future SDK versions that would be possible without Speaker.cs modification, via factory as it is now for AudioIn.

  • I need an event so we can control the fmod side of it (attenuarion, track for volume etc)

  • I may have spoken too soon.

    Fmod did work for me when I tested two clients running locally on PC, but I had to increase the programmer instrument to be as long as 15 seconds to make it work reliably.

    However, when running two clients on Android, the sound is still cutting out for us about every 3 seconds (suspiciously similar to the 3 second long buffer created by AudioOutDelayControl).

    Furthermore, we control the volume of the voice in game using a fmod VCA. If I drag that all the way down to 0 and then back up again, the sound gets all garbled.

    Neither of these issues happened with the default Unity AudioSource output.

    Any ideas?

  • Some debugging showed that it was the fmod AudioInput that didnt work well on android. We will use Photon Microphone instead as a work around.

    The volume issue seems to have been from having Envelope=Auto on the programmer instrument in fmod. Setting it to off solved the garbled sound issue.

  • Thanks for sharing this.

  • After testing for a few weeks, we still get some issues with this setup. Once in a while, the audio is turning out scrambled for one of the participants.

    Any idea what could be causing this?

  • gorrj
    gorrj
    edited March 7

    @Bexelius We are also trying to get this working and are encountering the same garbled audio issue. One person I can hear fine but they can't understand anything that I'm saying. We'll be trying to resolve this this week.

  • @gorrj Did you check the Envelope setting on the programmer instrument in fmod studio? Setting that to off made volume work for us, though it still turns out scrambled randomly.

    I reviewed the photon code and saw that the buffer used is delay max (1000ms default) * 3 (hardcoded). But the actual seek will be from a point modulo that length, and then so much longer forward, so Im thinking an instrument 2x that length would be needed (ie 6 seconds). Could use some official guideline on this though…

  • I noticed when debugging this that the constructor in Photon.Voice.FMOD.AudioOut was using processInService: false for the base class constructor, while the old UnityAudioOut was using processInService: true. In other words, the fmod output used a code path that hasnt really been used before, and thus susceptible to bugs. I changed it to true for fmod too, and it still sounds good. Will try this for a while and see if it solves the issue...

  • gorrj
    gorrj
    edited March 8

    @Bexelius Our programmer instrument did not have an envelope. We can play with the track length but it was already set to 15 seconds. Thank you for that observation.

  • Bexelius
    Bexelius
    Answer ✓

    None of my attempts have helped to remove the occacional garbled sound. Furthermore, I also found an issue where creating new speakers to occacionally failed to create an fmod sound.


    None of these issues happen when using UnityAudio, so we have decided to abandon this effort and revert to using the default UnityAudio output.

  • Thank you for creating this thread @Bexelius! Any luck fixing the garbled sound issue so far? Is it worth spending time trying to implement?

    This seems to literally be the only web page about using Photon Voice with FMOD. Would you have any more information to share with us @JohnTube?

  • vadim
    vadim mod
    Answer ✓

    FMOD support in Photon Voice is experimental. Its main purpose is to demonstrate integration of 3rd party libraries with the Voice. Both FMOD.AudioOut and FMOD.AudioOutEvent work fine in my tests. But maybe my setup is different from what real FMOD users have. My experience in FMOD is limited. I just made it working in my simple test.

    Can you reproduce the issue with DemoVoiceUI or other demo from Photon Voice?

    For FMOD.AudioOutEvent, make sure that:

    • the event contains "Programmer Instrument"
    • Programmer Instrument is looped
    • the event is persistent
    • probably the event should be assigned to a bank
    • fmod project is specified in unity fmod settings
    • don't forget to run "build" in fmod studio after modifications


  • @vadim Thanks for your reply.

    I'm getting errors right after enabling the PHOTON_VOICE_FMOD_ENABLE define:

    • FMODAudioIn.cs(5,17): error CS0246: The type or namespace name 'FMOD' could not be found
    • FMODAudioInEnumerator.cs(4,17): error CS0246: The type or namespace name 'FMOD' could not be found
    • FMODAudioOut.cs(5,17): error CS0246: The type or namespace name 'FMOD' could not be found

    Rider shows FMOD in red here:

    using FMODLib = FMOD;
    

    The FMOD plugin is installed in the project and works as expected. I am not a namespace person and I couldn't figure it out. Any idea? Thanks.

  • Well that didn't take long. Immediately after sending the above message, I tried to add FMODUnity to PhotonVoice.API asmdef and that made the trick.

  • References to 3rd party dependencies may be missing or invalid in the current Photon Voice package. We will look into this.