Player joining match sees old destroyed scene objects?

Hello, I'm running into a strange bug and I'd like to see if there's a way to solve it.

We've got a lobby/match setup almost identical to Call of Duty 4. Players press "quick match" and join/create a random game, then sit in a room lobby until enough players have joined and until the various intermission timers have finished. The match then starts by loading the map set by the room. Once the scene is loaded, several scene objects are spawned by the MasterClient to handle things like gametype, score, etc. Once the map has ended and shown the scoreboard, the match loads back out to the main menu. The main menu sees that the players are in a room and jumps them back to the room lobby menu and the timers start again. All of this works great!

A healthy match (log is from client):





The problem is that when a player joins a room lobby still in the main menu but after a match has ended in that room, they load the gametype/score/etc scene objects spawned in the previous game. These objects don't exist on the host machine, but do somehow update themselves as if they were (all of the update logic is hidden behind PhotonNetwork.isMasterClient).

Joining a match in the lobby after a game has concluded:



A diagram to better explain when specifically this is happening:



If a player joins a room that's in the game scene these leftover objects aren't loaded and the appropriate ones are. If a player waits out a bugged lobby they'll load into the new game scene no problem, and when exiting the scene will have the scene objects properly disposed.

Here's how I load into the scene:

private void LoadMatch()
{
	Debug.LogFormat("Loading {0} on {1}", gametype.ProperName, map.ProperName);
	PhotonNetwork.LoadLevel(map.SceneName);
}

Here's how I spawn the scene objects:

/// <summary>
/// Creates a team score object on the network.
/// </summary>
/// <param name="team">Team.</param>
private void CreateTeamScore(Team team)
{
	if (teamScorePrefab)
	{
		Debug.LogFormat("Created TeamScore for team {0}.", team);
		PhotonNetwork.InstantiateSceneObject(teamScorePrefab.name, Vector3.zero, Quaternion.identity, 0, teamScorePrefab.GetInstantiationData(team));
	}
}

Here's how I load out of the scene:

else if (state == State.Finished)
{
	state = State.Loading;
	PhotonNetwork.LoadLevel(lobbyScene); // FIXME: Do we use this, or do we manually go to our own lobby scene using something else? Will this bring along our child clients?
}

PhotonNetwork.automaticallySyncScene is set to true.