Effectively destroy objects

Options
Hi,
I have a new problem. I have a lobby where players can play with their characters, once all of them are ready, I load the game scene. The probleme is that I try to destroy the client's player gameobject with PhotonNetwork.Destroy() and then load the scene. But if there are multiple clients connected, the network destroy is received after the scene is loaded on a client and it tries to destroy an object that has been already destroyed by the scene change (by Unity itself), well I suppose that's what's happening.

So I get this error :

Ev Destroy Failed. Could not find PhotonView with instantiationId 2001. Sent by actorNr: 2

How can I handle the scene change and network objects destroy ? I could hardcode something for the players gameobject, check if there is 0 of them then load the scene but this problem will come back later with every other objects that will be instantiated.