A system to spawn players

Options
I've been trying to make a system for days now and all my attempts have failed.

The system: When players all load into the game, I have all their loading synced and it works fine.

I want to take that number of players and spawn in them in with 1 starting villager an 1 towncenter.

The system I have:

There is a array of objects called spawns and it goes up to 8. These objects have a script on them to check PhotonNetwork.Players.count, and matches the player when that number and spawns them.

The problems with this so far is that the players instantiate more than once. I fixed this by renaming the objects that the player is supposed to own to his name. And then I did a foreach loop for everyobject that isnt my player name and delete them. This deletes the objects correctly.

This system worked so far until I wanted the players to move their villagers. The movement works for the player only even tho I have photonview and tranform view. The system sends "We have no such photon view" and the movement doesnt sync.