Weird 2D sync

FBI

I'm programming the network movement for a simple bomberman game. Currently, I'm only moving a sprite across the bomberman grid which I created using a tilemap. Client-side player movement and collisions work perfectly. I've been play-testing the networking movement with a friend and realized something strange when the position is synchronized. This is a little bit tricky to describe, but I'll give it my best:

Player 1 stands at the very center. This position is perfectly synchronized over the network.
Now imagine Player 1 walking to the top, increasing the y position. The y position over at my friend's view will be slightly higher than my actual position. The difference of our player's y values increases the further I move up. And this goes for all directions.
If player 1 goes to the right (x value is increased) the x position at my friends's view will be slightly higher than mine. The difference also increases the further I move to the right.
(Here's an attempt to illustrate the problem: https://www.imageupload.co.uk/image/EOM9)

It's as if from the center outwards, the player is always a bit further in that direction. Does anyone know what to do?

This is my PlayerMovement script:

    private PhotonView PhotonView;
    private Vector3 TargetPosition;

    [SerializeField]
    private float speed; // Set the value in the inspector in the NewPlayer prefab

    private void Awake() {
        PhotonView = GetComponent();
    }

    private void FixedUpdate() {
        if(PhotonView.isMine) {
            MovePlayer();
        } else {
            SmoothMove();
        }
    }

    private void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info) {
        if(stream.isWriting) {
            stream.SendNext(transform.position);
        } else {
            TargetPosition = (Vector3)stream.ReceiveNext();
        }
    }

    private void MovePlayer() {
        float vertical = Input.GetAxisRaw("Vertical"); // x-Achse
        float horizontal = Input.GetAxisRaw("Horizontal"); // y-Achse

        transform.position += transform.up * (vertical * speed * Time.fixedDeltaTime);
        transform.position += transform.right * (horizontal * speed * Time.fixedDeltaTime);
    }

    private void SmoothMove() {
        transform.position = Vector3.Lerp(transform.position, TargetPosition, Time.fixedDeltaTime * 8);
    }