Measuring next position, extreme jumpback

Hi, I'm working on a custom sync script, and am trying to work out the kinks at the moment. I have a pretty decent setup so far, but I'm not sure how to improve what I got. I have a car mechanic, where multiple players can enter a vehicle. The driver becomes the owner and can drive the others inside around.

First, I tried the Transform and Rigidbody views, but the syncs for the passengers were just way off from what I was wanting (considering that passengers can't have ownership if the driver has it). After some research, I had created my custom sync script that would essentially do the same thing, but synced with input reads from the driver. The result is better, but there are areas where it goes back to jumping around. My current stream function looks like this:

public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info)

  {

    if (stream.IsWriting)

    {

/// x = throttle, y = steerInput, z = brake ///

      stream.SendNext(x);

      stream.SendNext(y);

      stream.SendNext(z);

      stream.SendNext(pos);

      stream.SendNext(rot);

      stream.SendNext(currentSpeed);

    }

    else

    {

      x = (float)stream.ReceiveNext();

      y = (float)stream.ReceiveNext();

      z = (bool)stream.ReceiveNext();

      pos = (Vector3)stream.ReceiveNext();

      rot = (Quaternion)stream.ReceiveNext();

      currentSpeed = (float)stream.ReceiveNext();

      cc.RecieveInputs(x, y, z);

      checkDistance(pos, rot, currentSpeed);

    }

  }

public void checkDistance(Vector3 dis, Quaternion rt, float speed)

  {

    if (cc.enabled == false) return;

    float dist = Vector3.Distance(dis, transform.position);

    float ang1 = Quaternion.Angle(rt, transform.rotation);

    if (ang1 < 1) ang1 = 1;

    transform.rotation = Quaternion.Lerp(transform.rotation, rt, ang1 * Time.deltaTime);

    if (dist < distanceLimit) return;

    //Debug.Log(ang1);

    transform.position = Vector3.Lerp(transform.position, dis + (transform.forward * speed * timeBetweenFrameSync), dist * Time.deltaTime);

    timeBetweenFrameSync = 0;

  }

void LateUpdate()

  {

    timeBetweenFrameSync += 1 / 10;

  }

There's quite a bit more, but this piece is what I'm wanting to look at. the checkDistance script is my formula for measuring the car going forward, when the inputs start falling out of sync. I had added a variable timeBetweenFrameSync to measure how long a message took in frames, then tried to apply it. It runs fairly smooth in areas, but in others the car jumps backward in an extreme way. I would understand the jump motion forward, considering my code is just guessing the direction of the car, but I'm not seeing why the car would be jumping backwards.

I realize I have a bit of a packed question, I'm still new to networking and trying to understand what's going on. My next move would have been to look at the velocity. But in terms of the position change shown, would there be a reason for the extreme jumpback?