Photon Ownership Transfer Issues

Options
Hi all,

I've a particular issue that's been plaguing us since the beginning of our project. We are working in Unity3D, PhotonVoice 1.13.

Anytime an interactable object spawns in the game, the MasterClient spawns it. Then any player, if the interactable object allows, can grab it.

I wrote conditionals checking on the player type of the item. If the item belongs to no one, anyone who touches it first will make it theirs. So if player one touches it, I set the player type to Player One and transfer the ownership to that photon player. If Player Two tries to touch the item, the item on their end checks that it belongs to Player One, and disallows them from transferring ownership and thus not allowed to grab it. If Player One lets go of the item and the item is not destroyed, it returns to it's no one state, therefore letting Player Two pick up the item if they want. The player type is being synced via OnPhotonSerializeView. The interactable objects are currently in Takeover state.

This works well enough when it's one player at a time. It's when players try to grab the same object at the same time or when they try to "trade" items where we encounter issues.
example. Player One and player two grabs a hammer at the same time. It says that the hammer belongs to Player One, but the hammer is "jittering" between player one's and player two's hand.

Or when Player One lets go of an item, the item shows that it is no one's player type. But when Player Two touches it, owns it, and grabs it, Player One sees a "bobbing" item in Player Two's hand, rather than the item following Player Two's hand accurately.

I'm using PhotonTransformView to sync the vectors, so any items on the ground shouldn't be moving and any items in a player's hand should be moving with them.

What can I do in this case?