[PUN2] NavAgent + PhotonTransformClassic teleports when host migrates

We have a system to migrate host to someone with better ping and fps. We also have AI nav agent spawned by the Master Client using scene instantiate. The AI is moved on the master client's side. However when there is host migration it seems to teleport to another location. Anyone else faced this issue? Anyone thinks I can try something here?