NullReferenceException from EntityChannel+CommandChannel.ReadInput

Hi all,

In my project I'm trying to sync player positions via commands.

Recently I've been getting random disconnects from clients (with no console errors other than the following two)

I'm getting the following error randomly on my headless server:
NullReferenceException: Object reference not set to an instance of an object
EntityChannel+CommandChannel.ReadInput (Bolt.Packet packet) (at <7178d3d2fcc34efc981974cff1b135f8>:0)
EntityChannel+CommandChannel.Read (Bolt.Packet packet) (at <7178d3d2fcc34efc981974cff1b135f8>:0)
BoltConnection.PacketReceived (UdpKit.UdpPacket udpPacket) (at <7178d3d2fcc34efc981974cff1b135f8>:0)
UnityEngine.Debug:LogException(Exception)
BoltLog:Exception(Exception)
BoltConnection:PacketReceived(UdpPacket)
BoltInternal.BoltCore:Udp_PacketReceived(UdpEvent)
BoltInternal.BoltCore:PollNetwork()
BoltInternal.BoltCore:Poll()
BoltPoll:FixedUpdate()

accompanied by:
Exception thrown while unpacking data from [EndPoint IPv4 192.168.1.233:51880], disconnecting
UnityEngine.Debug:LogError(Object)
UnityWriter:BoltLog.IWriter.Error(String)
BoltLog:Error(String)
BoltLog:Error(String, Object)
BoltConnection:PacketReceived(UdpPacket)
BoltInternal.BoltCore:Udp_PacketReceived(UdpEvent)
BoltInternal.BoltCore:PollNetwork()
BoltInternal.BoltCore:Poll()
BoltPoll:FixedUpdate()

I'm guessing "ChannelCommand" implies something is null in my PlayerCommand that I'm trying to send and sync, but when logging every single property on the server and client I couldn't find any null properties being packed into the command.

Any ideas/further information on what may be causing these errors would be great!

Comments

  • Hello @JoshPT ,

    Make sure you've recompiled Bolt ("Bolt->Compile Assembly") after any changes to your Bolt Assets (commands, events, state), so your peers do not serialize your data on any different order.

    Which Bolt version are you using?

    --
    Ramon Melo
    Photon Bolt Team
  • Thanks for the quick reply @ramonmelo

    I have ran Compile Assembly and my server/client are on the exact same copy of the game. project.json and Bolt.user.dll are identical on the two connected machines (shared over version control)

    I'm running Bolt 1.2.13c

    I'm sending an IProtocolToken with an array in my PlayerCommand if that could be causing the issue?

    I assume the issue is caused by something I'm doing wrong but the error isn't super helpful so I'm not exactly sure what I'm doing wrong. The fact the client can play but gets disconnected seemingly at random times makes it even more confusing! Any more info would be great
  • Hello @JoshPT ,

    Yes, that may explain the error if you are not (de)serializing your Token properly.

    Can you show your Token Write/Read methods?

    Also, if you have a reproduction project showing the issue, we are glad to take a look.
    Send us an email with the project to
    developer@photonengine.com
    

    --
    Ramon Melo
  • Hi @ramonmelo

    My Write/Read methods look like this:
        public void Read(UdpPacket packet)
        {
            Frame = packet.ReadInt();
            RootPosition = packet.ReadVector3(-8192, 8192, 0.01f);
            RootRotation = packet.ReadQuaternion(0.01f);
            HeadPosition = packet.ReadVector3(-8192, 8192, 0.01f);
            HeadRotation = packet.ReadQuaternion(0.01f);
            HandLeftPosition = packet.ReadVector3(-8192, 8192, 0.01f);
            HandLeftRotation = packet.ReadQuaternion(0.01f);
            HandRightPosition = packet.ReadVector3(-8192, 8192, 0.01f);
            HandRightRotation = packet.ReadQuaternion(0.01f);
        }
    
        public void Write(UdpPacket packet)
        {
            packet.WriteInt(Frame);
            packet.WriteVector3(RootPosition, -8192, 8192, 0.01f);
            packet.WriteQuaternion(RootRotation, 0.01f);
            packet.WriteVector3(HeadPosition, -8192, 8192, 0.01f);
            packet.WriteQuaternion(HeadRotation, 0.01f);
            packet.WriteVector3(HandLeftPosition, -8192, 8192, 0.01f);
            packet.WriteQuaternion(HandLeftRotation, 0.01f);
            packet.WriteVector3(HandRightPosition, -8192, 8192, 0.01f);
            packet.WriteQuaternion(HandRightRotation, 0.01f);
        }
    

    The ReadVector, WriteVector, ReadQuaternion & WriteQuaternion methods are custom methods which compress the values within the token. They all get compressed to floats (and ints) through these methods:

        public static float ReadFloat(this UdpPacket packet, float minValue = float.MinValue, float maxValue = float.MaxValue, float precision = 0.0000001f)
        {
            int intMax = (int)((maxValue - minValue + precision) * (1f / precision));
            int intVal = packet.ReadInt(0, intMax);
            float value = (intVal * precision) + minValue;
            value = Mathf.Round((value) * (1f / precision)) * precision;
            return value;
        }
    
        public static void WriteFloat(this UdpPacket packet, float value, float minValue = float.MinValue, float maxValue = float.MaxValue, float precision = 0.0000001f)
        {
            value = Mathf.Clamp(value, minValue, maxValue);
            int intMax = (int)((maxValue - minValue + precision) * (1f / precision));
    
            bool neg = (value < 0);
            value = value - minValue;
    
            value = !neg ? Mathf.Floor(value * (1 / precision)) * precision : Mathf.Ceil(value * (1 / precision)) * precision;
            
            float intVal = value * (1f / precision);
    
            packet.WriteInt((int)intVal, 0, intMax);
        }
    
        public static int ReadInt(this UdpPacket packet, int minValue = int.MinValue, int maxValue = int.MaxValue)
        {
            int reqBit = (int)RequiredBits(minValue, maxValue);
            return packet.ReadInt(reqBit) + minValue;
        }
    
        public static void WriteInt(this UdpPacket packet, int value, int minValue = int.MinValue, int maxValue = int.MaxValue)
        {
            int reqBit = (int)RequiredBits(minValue, maxValue);
    
            value = Mathf.Clamp(value, minValue, maxValue);
            value = value - minValue;
    
            packet.WriteInt(value, reqBit);
        }
    
        public static uint RequiredBits(int minValue, int maxValue)
        {
            uint delta = (uint)(maxValue - minValue);
    
            uint x = delta;
            x |= x >> 1;
            x |= x >> 2;
            x |= x >> 4;
            x |= x >> 8;
            x |= x >> 16;
            x -= x >> 1 & 0x55555555;
            x = (x >> 2 & 0x33333333) + (x & 0x33333333);
            x = (x >> 4) + x & 0x0f0f0f0f;
            x += x >> 8;
            x += x >> 16;
    
            return x & 0x0000003f;
        }
    

    Unfortunately the project is a large company project so I wouldn't be allowed to send it. Apologies

    Thanks
  • First thing to do is start using basic debugging techniques.

    Start with binary searching. Remove all the compression in your packing calls. My first inclination is something might be wrong with one of those compression methods. Remove those as a variable and see if you can repro. If you cannot, then probably add back in just the vectors and see if it happens, and so on until you narrow down the issue.

    Also keep in mind that a single bit mistake in any token across the packet will cause catastrophic failures. Sometimes you might get an exception in this token call above, but it might have been caused by something before it in the packet. Usually these mistakes manifest pretty quickly as an exception, but sometimes the packet unpack can continue for a bit before you get an exception somewhere.

    Also, separate read/write methods should immediately become obvious are a nightmare. A single mistake will cause everything to die. If something like this happened it would be a nightmare to track down. Honestly, the first thing you should do is abstract the packet read/write methods so you can call a single read/write method, probably as inlined extension methods, that uses the same parameters and use that for both read and write.
  • Same here ! After random amount of time (sometimes right after connect, sometimes after 5-10 minutes of game play) server kicks client with error:

    I have simple command input:
        public override void SimulateController()
        {
            IENDCharacterContolCommandInput input = ENDCharacterContolCommand.Create();
            input.MoveOrigin = moveOriginal; //Vector3
            input.Fire = _characterWeapon.TriggerPressed; //bool
            input.Torso = _torsoBoneRotator.TorsoRotation; //float
            if(_characterWeapon.HasWeapon())
                input.ProjectileOrigin = _characterWeapon.GetProjectileOriginFromPrefab(); //Vector3
    
            entity.QueueInput(input);
        }
    

    Unknown [TypeId:68] //<- random number here
    UnityEngine.Debug:LogError(Object)
    UnityWriter:BoltLog.IWriter.Error(String)
    BoltLog:Error(String)
    BoltLog:Error(String, Object)
    Bolt.Factory:Create(TypeId)
    Bolt.Factory:NewCommand(TypeId)
    CommandChannel:ReadInput(Packet)
    CommandChannel:Read(Packet)
    BoltConnection:PacketReceived(UdpPacket)
    BoltInternal.BoltCore:Udp_PacketReceived(UdpEvent)
    BoltInternal.BoltCore:PollNetwork()
    BoltInternal.BoltCore:Poll()
    BoltPoll:FixedUpdate()
    

    then
    NullReferenceException: Object reference not set to an instance of an object
    EntityChannel+CommandChannel.ReadInput (Bolt.Packet packet) (at <7178d3d2fcc34efc981974cff1b135f8>:0)
    EntityChannel+CommandChannel.Read (Bolt.Packet packet) (at <7178d3d2fcc34efc981974cff1b135f8>:0)
    BoltConnection.PacketReceived (UdpKit.UdpPacket udpPacket) (at <7178d3d2fcc34efc981974cff1b135f8>:0)
    UnityEngine.Debug:LogException(Exception)
    BoltLog:Exception(Exception)
    BoltConnection:PacketReceived(UdpPacket)
    BoltInternal.BoltCore:Udp_PacketReceived(UdpEvent)
    BoltInternal.BoltCore:PollNetwork()
    BoltInternal.BoltCore:Poll()
    BoltPoll:FixedUpdate()
    

    Command settings:

    Pbf6PHg.png


    Unfortunately i cant send project cause it is under NDA. Current Bolt version: Bolt Free Debug v1.2.13.c
  • Hello @Muxeu ,

    Can you try with "Enable Delta Compression" disabled on the command?

    It may have an issue with the delta compression, and as your command is simple and you are already using the compression settings, that should not be a problem. We are investigating if there is really a problem with the Delta Compression implementation.

    --
    Ramon Melo
    Photon Bolt Team
  • ramonmelo wrote: »
    Hello @Muxeu ,

    Can you try with "Enable Delta Compression" disabled on the command?

    It may have an issue with the delta compression, and as your command is simple and you are already using the compression settings, that should not be a problem. We are investigating if there is really a problem with the Delta Compression implementation.

    --
    Ramon Melo
    Photon Bolt Team

    Hi ! Yes, disabling 'Delta Compression' fixed the problem. I found 100% reproduce steps:

    1. Enable 'Delta Compression' for command input.
    2. Build standalone game.
    3. Set bolt setting lag simulation like 200 ping and 5-10% packets loss
    4. Run Editor as Server with lag simulation.
    5. Run standalone build as client.
    6. Receive error after coupe of seconds.
  • Hello @Muxeu ,

    Thanks for your feedback, we will take a better look at the implementation.
  • It feels like this is basically what I said a month ago. Is this thing on? *tap* *tap* :-)
  • For some reason, rebuilding the project worked for me.

    I was not having the same issue but similar. Instead of "EntityChannel+CommandChannel.ReadInput" I had: "EntityChannel.ReadUpdate" and at <7178d3d2fcc34efc981974cff1b135f8>:0, same as first problem.