Could use some help figuring out bouncy physics material in this PhotonEngine game

Options
So I'm trying to prototype a primitive 1v1 fighting game where the goal is to push the opponent off of the platform rather than actually fight. This is working great in a local multiplayer setup using a physics material on the "belly" of the Rigidbody2d players (the circle collider gameobject nested inside of the square gameobject has the bouncy material).

But I'm toying with the idea of making it a network game and it seems the physics material doesn't get communicated too well over the Photon Realtime connections?

Here is a video showing how it seems like sometimes the position is updated but sometimes a player runs into another player and its like a brick wall:

https://imgur.com/QYWH178

Any ideas of what to try or if this is even possible?

A bit about how things are currently setup:
Each player object has the nested collider with a high bounciness material on it. Each player moves via the Rigidbody2D component via

.AddForce(direction * Speed);
I have a `Photon View` component on the player prefab and a `Photon Rigibody 2D View` component that is checked to Synchronize Velocity and Angular Velocity, as well as a `Photon Transform View` component synchronizing Position. Both the transform view and Rigidbody2dView are Observed Components in the Photon View.

I also found a script someone posted where they were manually sending the position/velocity data via `OnPhotonSerializeView()` I've tried adding that script and making that observable as well, but no luck, but it looks something like this:

public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info)
{
if (stream.IsWriting)
{
//We own this player: send the others our data
stream.SendNext(transform.position);
stream.SendNext(transform.rotation);
stream.SendNext(r.velocity);
stream.SendNext(r.angularVelocity);
}
else
{
//Network player, receive data
latestPos = (Vector3)stream.ReceiveNext();
latestRot = (Quaternion)stream.ReceiveNext();
velocity = (Vector2)stream.ReceiveNext();
angularVelocity = (float)stream.ReceiveNext();

valuesReceived = true;
}
}

// Update is called once per frame
void Update()
{
if (!photonView.IsMine && valuesReceived)
{
//Update Object position and Rigidbody parameters
transform.position = Vector3.Lerp(transform.position, latestPos, Time.deltaTime * 5);
transform.rotation = Quaternion.Lerp(transform.rotation, latestRot, Time.deltaTime * 5);
r.velocity = velocity;
r.angularVelocity = angularVelocity;
}
}

Answers