Undocumented breaking change in StreamBuffer (upgrading PUN 1.92 -> 1.96)

rhys_vdw

Hi there, we've encountered a breaking change when upgrading from 1.92 to 1.96. Unfortunately I can't work out how browse code in the releases between these two via the asset store, but I'm sure the developers will know what I'm talking about.

We relied on ExitGames.Client.Photon.StreamBuffer inheriting from System.IO.Stream. Our message handling is wired up using this helper class:

public class Message {
    readonly Serializer _serializer;
    public readonly byte Id;

    public Message(MessageId id, Serializer serializer) {
        PhotonPeer.RegisterType(
            typeof(T),
            (byte) id,
            Serialize,
            Deserialize
        );
        Id = (byte) id;
        _serializer = serializer;
    }

    public void Send(T payload, bool sendReliable, RaiseEventOptions options) {
        PhotonNetwork.RaiseEvent(Id, payload, sendReliable, options);
    }

    short Serialize(StreamBuffer outStream, object obj) {
        var writer = new BinaryWriter(outStream);
        var prevLength = outStream.Length;
        _serializer.Serialize(writer, (T) obj);
        return (short) (outStream.Length - prevLength);
    }

    object Deserialize(StreamBuffer inStream, short length) {
        var reader = new BinaryReader(inStream);
        return _serializer.Deserialize(reader);
    }
}

This relies on the ability to pass the StreamBuffer to both BinaryWriter and BinaryReader. Somewhere between PUN 1.92 and 1.96 StreamBuffer was changed such that it no longer inherits from System.IO.Stream.

I came up with this solution because I couldn't work out how to serialize strings through the RaiseEvent method, and instead leaned on C#'s BinaryWriter which has this functionality out of the box.

For example:

public class PlayerSerializer : Serializer {
    public override void Serialize(BinaryWriter writer, Player instance) {
        writer.Write(instance.Name);
        writer.Write((byte) instance.PhotonPlayerId);
        writer.Write((byte) instance.LocalInputId);
        writer.Write(instance.IsReady);
        writer.Write(instance.IsJoined);
        writer.Write((byte) instance.TeamId);
        writer.Write((byte) instance.MaterialId);
        writer.Write(instance.ClientId);
    }

    public override void Deserialize(BinaryReader reader) {
        var instance = new Player();
        instance.Name = reader.ReadString();
        instance.PhotonPlayerId = reader.ReadByte();
        instance.LocalInputId = reader.ReadByte();
        instance.IsReady = reader.ReadBoolean();
        instance.IsJoined = reader.ReadBoolean();
        instance.TeamId = reader.ReadByte();
        instance.MaterialId = reader.ReadByte();
        instance.ClientId = reader.ReadString();
        return instance;
    }
}

This is all working perfectly in 1.92. Is Stream support likely to be restored in future? If not, is there a guide/example for serializing complex objects, arrays and strings?

Thanks.

JohnTube

Hi @rhys_vdw,

Thank you for choosing Photon!

You can switch to the new "Byte Array Methods" as documented here.
You can also take a look at "Assets\Photon Unity Networking\Plugins\PhotonNetwork\CustomTypes.cs".

rhys_vdw

Hi JohnTube, thanks for the response.

Hm. Just so I understand, you're suggesting I do the following:

Current serialization method:
1. Get payload object
2. Serialize into Photon's stream

Proposed serialization method (I think?):
1. Get payload object
2. Create a MemoryStream
3. Serialize the payload into the memory stream
4. Convert the MemoryStream into a byte[] with ToArray()
5. Pass it to Photon so that it can be copied into its Stream (assume this is what's going on under the hood?)

This seems like a lot of unnecessary steps. Is there a way to do it with fewer heap allocations? Also does the byte array method mean that we're getting an extra array creation when deserializing? I'm not completely across how these streams work but I chose the stream API to minimize allocations.

rhys_vdw

Also, how do I enable email notifications for this forum? I didn't get anything when you replied.

rhys_vdw

nvm, I found the option... maybe I disabled it.

JohnTube

Hi @rhys_vdw,

Sorry I misread some information.
I got some details from my colleague @Tobias who knows more about this since he is the lead developer of PUN. Unfortunately, he is on sick leave now.

the ability to pass the StreamBuffer to both BinaryWriter and BinaryReader

Yes this was removed on purpose.

Is Stream support likely to be restored in future?

"the Stream compatibility is not coming back. it kept us from accessing the byte[] directly, which is sometimes a benefit over the stream."

If not, is there a guide/example for serializing complex objects, arrays and strings?

Maybe @jeanfabre can help in the meantime.

rhys_vdw

Thanks @JohnTube. The advantage of having a stream based API is that we could serialize arbitrarily sized objects (primary offender is strings, e.g. player names), without knowing how much memory to allocate ahead of time. This allows us to do this kind of thing:

    public class GameStateSerializer : NewSerializer {
        MonkStateSerializer _monkStateSerializer;
        BlockStateSerializer _blockStateSerializer;
        public GameStateSerializer(
            MonkStateSerializer monkStateSerializer,
            BlockStateSerializer blockStateSerializer
        ) {
            _monkStateSerializer = monkStateSerializer;
            _blockStateSerializer = blockStateSerializer;
        }

        public override void Serialize(BinaryWriter writer, GameState gameState) {
            writer.Write(gameState.GameStateVars.FrameNumber);
            writer.Write(gameState.GameStateVars.GameStartTime);
            writer.Write(gameState.GameStateVars.GameEndTime);
            writer.Write(gameState.GameStateVars.LastLaserTurnOffTime);
            for (int i = 0; i < GlobalConfig.TeamCount; i++) {
                writer.Write((byte) gameState.GameStateVars.TeamScores[i]);
                writer.Write(gameState.GameStateVars.TeamScoreProgress[i]);
            }
            _monkStateSerializer.SerializeArray(
                writer,
                gameState.MonkStates,
                gameState.MonkStates.Length
            );
            _blockStateSerializer.SerializeArray(
                writer,
                gameState.BlockStates,
                gameState.BlockStates.Length
            );
        }

Anyway, maybe @Tobias or @jeanfabre know how best to do this kinda thing? We're on our Quantum trial so we won't be updating PUN any time soon (or ever if things go well).

Tobias

Yes, the StreamBuffer does no longer inherit from Stream. This gives us direct access to get the underlying byte array, which means less copying and garbage.

With a little refactoring, this change should not affect you that much: Serialize into your own BinaryWriter and get the serialized data from it as byte array. This can be written to the StreamBuffer.

rhys_vdw

Thanks @Tobias, we've moved to Quantum now which sidesteps this problem entirely by a) having a much nicer stream interface, and b) not requiring state to be transmitted. :-)