Unity Physics got disabled by Fusion

Options

Can Fusion disable my physics? I am connecting to a player, play a game everything is fine. Go to the main menu and leave the party. Then when I try to load the offline game every logic that links with Unity Physics seems to be not working. Spawn a cube in the editor with a rigidbody it doesn't fall, and colliders don't affect onTriggerEnter(). Of course, if I restart a game everything is working just fine.