Authorative Movement: Snipping back on control release


I am using an authorative server-client setup. Any object within the scene is owned by the server. Clients can however request control and, if on control, they can move an object.
Movement is replicated via the Transform state of the object to other clients. The replication mode: "Everyone except Controller" is therefore chosen. Smoothing is set to interpolation.
I am using the following logic to update the transform on the server side:
public override void SimulateController()
    var input = MovementCommand.Create();
    input.Position = transform.localPosition;
    input.Rotation = transform.rotation;

  public override void ExecuteCommand(Command command, bool resetState)
    MovementCommand cmd = (MovementCommand)command;
    if(!entity.IsOwner || !cmd.IsFirstExecution) {
    transform.localPosition = cmd.Input.Position;
    transform.rotation = cmd.Input.Rotation;

This works perfectly well. However, once a client releases the control, the object will flicker. The reason for that lies in the logic of the State.Transform sharing. In detail, the following happens:

1. A client takes over control over object
2. Client moves the object
3. State.Transform on the server & all other clients will update accordingly. Important: the State.Transform of the controller client does not update (due to the replication mode (?))
4. Controller client releases the Object
5. (Recent) controller client receives server command and now updates object transform. Problem: This command still contains the old State.Transform (prior to starting the control) as the state was never updated during the control
6. Object snips back to the old position
7. (Recent) controller client receives server command (now: new position)
8. Object snips to new position

It really seems that there is only one "problematic" frame. Do you guys know if there might be any solution to avoid this flickering? So maybe just tell the server, "well, if you did not update the State.Transform during the control, just take the new one"?
Many thanks!