Unity iOS Crash when starting recorder

Hi,
My App crashes about 25% of the time when joining a room and starting a recording (instantiating a prefab containing a PhotonVoiceView component). If I don't start the recording everything is fine.

Xcode shows the crash in AudioIn.mm in function performRender, line 131:
cd.pushCallback(cd.pushHostID, (float*)data.bytes, (int)data.length/sizeof(float));

Using PUN2 (2.17) and Voice2 (2.13), Unity 2019.2.21f1

Any help appreciated, it's awesome when it works, would love to be able to enable voice!
Thanks

Comments

  • JohnTube
    JohnTube ✭✭✭✭✭
    edited March 2020
    Hi @Numa,

    Thank you for choosing Photon!

    Can we have more information about those 25% cases to narrow down the issue?

    Can we have logs before the crash or crash report?
    If the crash happens in XCode could you debug & get the parameters in pushCallback?

    What Photon Voice settings do you use?
    What device(s) and iOS version(s) does this happen on?

    Does this happen with DemoVoicePun scene?

    Do you always add a "Microphone Usage Description" to your project's "Player Settings" under "Other Settings" -> "Configuration" section?
    Alternatively, you must add a NSMicrophoneUsageDescription key to "Info.plist" in Xcode project exported by Unity. The string value should explain why your application uses microphone.
    Failure to do so may result in crashes.
  • Ok I will try all this and get back to you, thanks.
    Do you have any tips for debugging Unity + XCode? When it crashes it's pointing at that line but all the variables I try to inspect come up as nil even though they aren't.
  • Having all the same problems on iOS 13.3.1 and latest Photon Voice. Just cannot get it to work. Crashes with above when using Photon Microphone Type but at least the app doesn't crash when using Unity setting. However, in BOTH cases I'm not getting any audio coming across. Do we have to explicitly set a Microphone Device? Thought it would just take the iPhone/iPad default devices when it starts up. Have Auto Start and Transmit Enabled checked and Debug Echo mode on. Primary Recorder set in Photon Voice Network. What am I missing here?
  • Update:
    - Tested the demo scene, no crashes
    - Mic usage description is filled properly
    - When using Unity as the microphone type + ios force speaker script, no problem.
    - When using Photon as the microphone type, crash very frequently.
    - Not sure if it's related but we are joining the game via deeplink (app opens -> joins room -> OnJoinedRoom event is received then the recorder is initialised). Maybe a race condition there? We tried adding a lot of delay in between each steps but it didn't help.

    We could use the Unity type microphone, but the echo is really bad (we need to use the speakers).

    I can't seem to be able to debug that part in Xcode or get a meaningful crash log, any tips on this?

    Any help appreciated.
    Thanks
  • I will add that I did the same above as @Numa and got exactly the same result. On iOS 13.3.1, Unity 2019.30f5, and Voice 2.14 latest download. Crashes on both iPad and iPhone when using Photon microphone type.

    In Xcode I don't get a very good error either other than the code highlighted on:
    cd.pushCallback(cd.pushHostID, (float*)data.bytes, (int)data.length/sizeof(float));
    
    With a red "Thread 1: EXC:BAD_ACCESS" error.

    Everything is setup per the Voice w/ Pun instructions and similar to the demos. But additionally, I get NO audio whatsoever even when using the Unity Microphone Type. Have all the DebugEcho audio stuff setup and I created some extra code to set the UnityMicrophone Device at runtime just in case to first device [0] but still no luck. The Photon Voice stats shows some packets going back and forth but just nothing as far as audio. Been working on this for two days and it just doesn't work. At a loss...
  • JohnTube
    JohnTube ✭✭✭✭✭
    edited April 2020
    Hi @Grant, @Numa,

    Could you try importing this old iOS's AudioIn.mm and try to reproduce the crash?
  • Got it. Do we need to do anything with it or just import it? And is the for Photon mic device or also Unity? I can't get either to work.
  • Recompiled and ran with the new file and using Photon Mic device on Game config. Didn't seem to crash this time but getting these errors (which I've got before but can't remember which of a million different combinations I tried):
    2020-04-02 12:54:24.872821-0700 xxxxxxx[63304:18074525] Setting category = AVAudioSessionCategoryPlayAndRecord, mode = AVAudioSessionModeVoiceChat, options = 1
    2020-04-02 12:54:24.995936-0700 xxxxxxx[63304:18074525] [aurioc] AURemoteIO.cpp:1086:Initialize: failed: -66635 (enable 3, outf< 1 ch,  48000 Hz, Float32> inf< 1 ch,  48000 Hz, Float32>)
    2020-04-02 12:54:25.000756-0700 xxxxxxx[63304:18074184] Route change:
    2020-04-02 12:54:25.000823-0700 xxxxxxx[63304:18074184]      CategoryChange
    2020-04-02 12:54:25.004120-0700 xxxxxxx[63304:18074525] [aurioc] AURemoteIO.cpp:1086:Initialize: failed: -66635 (enable 3, outf< 1 ch,  48000 Hz, Float32> inf< 1 ch,  48000 Hz, Float32>)
    2020-04-02 12:54:25.005676-0700 xxxxxxx[63304:18074525] Error returned from setupIOUnit: couldn't initialize AURemoteIO instance (-66635)
    2020-04-02 12:54:25.023923-0700 xxxxxxx[63304:18074184]  New Category: AVAudioSessionCategoryPlayAndRecord
    2020-04-02 12:54:25.024090-0700 xxxxxxx[63304:18074184] Previous route:
    2020-04-02 12:54:25.024305-0700 xxxxxxx[63304:18074184] <AVAudioSessionRouteDescription: 0x2822d8e80, 
    inputs = (
        "<AVAudioSessionPortDescription: 0x2822d9120, type = MicrophoneBuiltIn; name = iPhone Microphone; UID = Built-In Microphone; selectedDataSource = (null)>"
    ); 
    outputs = (
        "<AVAudioSessionPortDescription: 0x2822d90c0, type = Receiver; name = Receiver; UID = Built-In Receiver; selectedDataSource = (null)>"
    )>
    2020-04-02 12:54:25.024357-0700 xxxxxxx[63304:18074184] Current route:
    2020-04-02 12:54:25.025978-0700 xxxxxxx[63304:18074184] <AVAudioSessionRouteDescription: 0x2822c13d0, 
    inputs = (
        "<AVAudioSessionPortDescription: 0x2822c15e0, type = MicrophoneBuiltIn; name = iPhone Microphone; UID = Built-In Microphone; selectedDataSource = Bottom>"
    ); 
    outputs = (
        "<AVAudioSessionPortDescription: 0x2822c12e0, type = Receiver; name = Receiver; UID = Built-In Receiver; selectedDataSource = (null)>"
    )>
    2020-04-02 12:54:25.708003-0700 xxxxxxx[63304:18074184] Route change:
    2020-04-02 12:54:25.708102-0700 xxxxxxx[63304:18074184]      CategoryChange
    2020-04-02 12:54:25.717472-0700 xxxxxxx[63304:18074184]  New Category: AVAudioSessionCategoryPlayAndRecord
    2020-04-02 12:54:25.717585-0700 xxxxxxx[63304:18074184] Previous route:
    2020-04-02 12:54:25.718246-0700 xxxxxxx[63304:18074184] <AVAudioSessionRouteDescription: 0x2822ce8e0, 
    inputs = (
        "<AVAudioSessionPortDescription: 0x2822cea70, type = MicrophoneBuiltIn; name = iPhone Microphone; UID = Built-In Microphone; selectedDataSource = Bottom>"
    ); 
    outputs = (
        "<AVAudioSessionPortDescription: 0x2822cef30, type = Receiver; name = Receiver; UID = Built-In Receiver; selectedDataSource = (null)>"
    )>
    2020-04-02 12:54:25.718381-0700 xxxxxxx[63304:18074184] Current route:
    2020-04-02 12:54:25.746276-0700 xxxxxxx[63304:18074184] <AVAudioSessionRouteDescription: 0x2822c56c0, 
    inputs = (
        "<AVAudioSessionPortDescription: 0x2822c5930, type = MicrophoneBuiltIn; name = iPhone Microphone; UID = Built-In Microphone; selectedDataSource = (null)>"
    ); 
    outputs = (
        "<AVAudioSessionPortDescription: 0x2822c50a0, type = Receiver; name = Receiver; UID = Built-In Receiver; selectedDataSource = (null)>"
    )>
    
  • Tried the new file and it all works great, no crash, no echo :)
    Thanks!

    Last question, the voice is clear but is crackling pretty often, any settings recommendations? Right now I'm using
    20ms frames
    24000 sampling
    30000 bitrate
    VAD (0.06)
    Mic amplifier (factor 10, boost 3)

  • Numa
    Numa
    edited April 2020
    Just tried the demo scene again (DemoVoicePun), looks like the problem is with the Photon type microphone:
    - Photon microphone: audio crackles a lot
    - Unity microphone: Perfect sound (but echo)
    - Photon microphone + original AudioIn.mm file: crash 100% on startup after 2-3 seconds.
    I'm using iOS 13.3.1 + Unity 2019.2.21f1

  • JohnTube
    JohnTube ✭✭✭✭✭
    edited April 2020
    Hey @Grant, @Numa,

    My colleague @vadim made a tremendous work investigating, fixing and optimizing this.
    It turns out that we made a change between 2.12 & 2.13 (missing in the change log, sorry) to move some logic in the AudioIn.mm plugin to a worker thread. This is what caused the crash...threading!

    So in order to not lose the performance gain which may result in slightly better audio quality using Photon MicrophoneType on iOS, I invite you to try out this new version with the optimized fix.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hey @Numa,
    - Photon microphone: audio crackles a lot

    If this persists after trying the new AudioIn.mm version, it could be caused by the MicAmplifier values.
  • JohnTube
    JohnTube ✭✭✭✭✭
    Hey @Grant,


    About the logs you shared:

    it looks like you are setting custom iOS AudioSessionSettings combination that does not work (Category + Mode + Options).
    Using custom settings is really advanced and should be last resort as it may waste a lot four time as it requires a lot of experimentation, tests and trial & error.
    So we recommend using presets VoIP or Game unless there is a better tested combination that works better.
  • Hi @JohnTube,

    I downloaded the new file and tried again. Same result and same errors listed above. And please note, I AM using the Game mode. I am not using custom audio settings. And again, neither Unity mic type nor Photon mic type returns sound. Ever. Unity doesn't crash as above but I just get nothing. I have a Recorder in my scene with Debug Echo on. Auto start and transmit checked. Speakers on all the avatars and Voice Views with setup debug speaker checked as well as Use Primary Recorder. Each Voice View's Speaker in Use set to each avatar's speaker component. What am I missing?? I cannot get any of this work.
  • JohnTube
    JohnTube ✭✭✭✭✭

    Hi @Grant,

    It looks like your issue is not related to iOS crash due to Photon MicrophoneType.
    So I suggest you create another forum discussion and try the demo scenes and see if you can reproduce.
    Also increase the log level of voice components and see if you can spot the issue there.

  • @JohnTube, I have run the demo scenes on both my devices and same thing. They do not work either. No sound whatsoever.
  • @JohnTube Thanks for the new file, it fixed my problem! And the audio crackling was fixed by selecting the VoIP mode instead of Game mode.
    Thanks for the quick help!