how to make any speech to text plugin work together with photon voice?

I am creating a multiplayer game using photon and also integrated photon voice to have voice chat with network players.
There is one chatbot functionality in my game where I'll ask some questions, chatbot coverts that speech to text and fetch the data from the backend.

But as I am already using Photon voice, its already using microphone. In this case when I am trying to enable my chatbot (putting him in listen mode), it does not get access to microphone.

Is it possible to have microphone access to both these plugins?
can someone help me with this?

Comments

  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @chetanmrane,

    Thank you for choosing Photon!

    Technically you don't need Photon Voice for this.
    You can just grab the voice input from the microphone using Unity's API.

    Usually, speech to text recognition requires is done server side, so you need a server for this.
    Photon does not provide this feature so you need to implement it or use a custom thrid party service for this.

    Are you using Photon Voice 1 or Photon Voice 2?

    If you want to get the input stream using Photon Voice, you can add a pre/post processor to get the recorded audio frames before they are sent from Photon Voice.

    Read an example about this for Photon Voice 1 or Photon Voice 2: § after "Second, outgoing voice streams:"

    both these plugins?
    What plugins exactly? if Photon Voice is one what is the other?
  • @JohnTube thanks for your reply

    In my multiplayer game, there is two separate feature I wanted to support.

    1. Players across networks can hear each other's voice - For this, I am using photon voice
    2. Chatbot functionality - Bot will recognize the player's speech and converts it into text - For this, I have created a plugin from Android native SpeechRecognizer class

    Now the problem I am facing is Photon voice and Speech to text cannot access the microphone at the same time.

    Is it possible in Unity to access Microphone by two separate plugins (in my case Photon voice and SpeechToText plugin)
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hi @chetanmrane,

    maybe use Photon MicrophoneType in Photon Voice and Unity's Microphone API in your SpeechToText plugin.
  • I've tried that already, it doesn't work with microphone type Photon or Unity... Any idea?