Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Welcome to our site

Take a moment to join our board

2Explosions

Linq.Expressions for packet serializing

Recommended Posts

Posted (edited)

Started a CO server emulator to spend time (internet down for ~3 weeks here, limited bandwidth on phone) and noticed how painful is to de/serialize packets in pretty much any open source emulator... well, i always thought that tbh., even in my previous projects.

So, as an excuse to learn a bit of System.Linq.Expressions, wich wasn't that hard as i thought, tried to avoid what the whole de/serialization code writing is. Keep in mind that this is still prototype code and before taking it any further, opinions would be be appreciated.

The code is based on two Type's properties iterator (tried to keep the code generic), the first one to access to the properties and the second to assign the values to the properties.

TypeIterator class:

Spoiler
public sealed class TypeIterator<TType, TAttribute, TIterator, TMethods>
        where TType : class
        where TAttribute : IterableAttribute
        where TMethods : class
    {
        public Action<TType, TIterator> AccessIteration { get; private set; }
        public Action<TType, TIterator> AssignIteration { get; private set; }

        private Dictionary<Type, string> aliases = new Dictionary<Type, string>();

        public TypeIterator(Dictionary<Type, string> typeAliases)
        {
            foreach (var kvp in typeAliases)
                aliases.Add(kvp.Key, kvp.Value);
            AccessIteration = CreateAccessIterator();
            AssignIteration = CreateAssignIterator();
        }

        private Comparison<Tuple<IterableAttribute, PropertyInfo>> comparer = (x, y) =>
        {
            return x.Item1.Position - y.Item1.Position;
        };

        private List<PropertyInfo> GetSortedProperties()
        {
            List<Tuple<IterableAttribute, PropertyInfo>> tuples = new List<Tuple<IterableAttribute, PropertyInfo>>();
            foreach (var property in typeof(TType).GetProperties())
            {
                IterableAttribute attribute;
                try
                {
                    attribute = property.GetCustomAttributes(typeof(TAttribute), false)[0] as IterableAttribute;
                    if (attribute == null)
                        continue;
                }
                catch { continue; }
                tuples.Add(new Tuple<IterableAttribute, PropertyInfo>(attribute, property));
            }
            tuples.Sort(comparer);
            List<PropertyInfo> result = new List<PropertyInfo>();
            foreach (var tuple in tuples)
                result.Add(tuple.Item2);
            return result;
        }

        private Action<TType, TIterator> CreateAccessIterator()
        {
            var instance = Expression.Parameter(typeof(TType));
            var iterator = Expression.Parameter(typeof(TIterator));
            List<Expression> calls = new List<Expression>();
            foreach (var property in GetSortedProperties())
            {
                object attribute;
                try
                {
                    attribute = property.GetCustomAttributes(typeof(TAttribute), false)[0];
                }
                catch { continue; }
                var constant = Expression.Constant(attribute);
                var methodInfo = typeof(TMethods).GetMethod("Accessor", new Type[] { typeof(TIterator), typeof(TAttribute), property.PropertyType });

                var access = Expression.Property(instance, property.Name);
                var call = Expression.Call(methodInfo, iterator, constant, access);
                calls.Add(call);
            }
            var body = Expression.Block(calls);
            var exp = Expression.Lambda<Action<TType, TIterator>>(body, instance, iterator).Compile();
            return exp;
        }

        private Action<TType, TIterator> CreateAssignIterator()
        {
            var instance = Expression.Parameter(typeof(TType));
            var iterator = Expression.Parameter(typeof(TIterator));
            List<Expression> calls = new List<Expression>();
            foreach (var property in GetSortedProperties())
            {
                object attribute;
                try
                {
                    attribute = property.GetCustomAttributes(typeof(TAttribute), false)[0];
                }
                catch { continue; }
                string alias;
                if (!aliases.TryGetValue(property.PropertyType, out alias))
                    alias = property.PropertyType.Name;
                var constant = Expression.Constant(attribute);
                var methodInfo = typeof(TMethods).GetMethod("Read" + alias, new Type[] { typeof(TIterator), typeof(TAttribute) });
                if (methodInfo == null)
                {
                    Console.WriteLine("cant find Get" + alias);
                }
                var access = Expression.Property(instance, property.Name);
                var call = Expression.Call(methodInfo, iterator, constant);
                var assign = Expression.Assign(access, call);
                calls.Add(assign);
            }
            var body = Expression.Block(calls);
            var exp = Expression.Lambda<Action<TType, TIterator>>(body, instance, iterator).Compile();
            return exp;
        }
    }

 

IterableAttribute class:

Spoiler
    [AttributeUsage(AttributeTargets.Property)]
    public class IterableAttribute : Attribute
    {
        public int Position { get; set; }
        
        public IterableAttribute(int position)
        {
            Position = position;
        }
    }

 

Usage:

PacketSerializer class

Spoiler
   public class PacketSerializer<TType> where TType : class
    {
        public static readonly Dictionary<Type, string> Aliases = new Dictionary<Type, string>()
        {
            { typeof(List<string>), "StringList" },
            { typeof(byte[]), "ByteArray" }
        };


        private TypeIterator<TType, PacketFieldAttribute, Packet, SerializationMethods> iterator;

        public PacketSerializer()
        {
            iterator = new TypeIterator<TType, PacketFieldAttribute, Packet, SerializationMethods>(Aliases);
        }

        public void Serialize(TType structure, Packet writer)
        {
            iterator.AccessIteration(structure, writer);
        }

        public void Deserialize(TType structure, Packet reader)
        {
            iterator.AssignIteration(structure, reader);
        }
    }

 

PacketFieldAttribute class

Spoiler
    public enum VariableLengthSize
    {
        None,
        OneByte,
        TwoBytes,
        FourBytes,
    }
    public class PacketFieldAttribute : FieldAttribute
    {
        public VariableLengthSize BytesForLength { get; set; }
        public int FixedSize { get; set; }

        public PacketFieldAttribute(int position) : base(position)
        {
        }
    }

 

SerializationMethods class, this one exposes what will happen to every property depending of it's type and attribute

Spoiler
public class SerializationMethods
    {
        public static void Accessor(Packet packet, PacketFieldAttribute attribute, short value)
        {
            packet.Write(value);
        }

        public static void Accessor(Packet packet, PacketFieldAttribute attribute, ushort value)
        {
            packet.Write(value);
        }

        public static void Accessor(Packet packet, PacketFieldAttribute attribute, string value)
        {
            if (attribute.FixedSize == 0)
            {
                switch (attribute.BytesForLength)
                {
                    case VariableLengthSize.OneByte:
                        packet.Encoding.GetByteCount(value);
                        packet.Write((byte)packet.Encoding.GetByteCount(value));
                        packet.Write(value);
                        break;
                    case VariableLengthSize.TwoBytes:
                        packet.Encoding.GetByteCount(value);
                        packet.Write((ushort)packet.Encoding.GetByteCount(value));
                        packet.Write(value);
                        break;
                    case VariableLengthSize.FourBytes:
                        packet.Encoding.GetByteCount(value);
                        packet.Write((uint)packet.Encoding.GetByteCount(value));
                        packet.Write(value);
                        break;
                }
            }
            else if (attribute.FixedSize > 0)
            {
                packet.WriteCString(value, attribute.FixedSize);
            }
            else
            {
                throw new InvalidOperationException();
            }
        }

        public static ushort ReadUInt16(Packet packet, PacketFieldAttribute attribute)
        {
            return packet.ReadUInt16();
        }

        public static short ReadInt16(Packet packet, PacketFieldAttribute attribute)
        {
            return packet.ReadInt16();
        }


        public static string ReadString(Packet packet, PacketFieldAttribute attribute)
        {
            if (attribute.BytesForLength > 0)
                switch (attribute.BytesForLength)
                {
                    case VariableLengthSize.OneByte:
                        return packet.ReadString(packet.ReadByte());
                    case VariableLengthSize.TwoBytes:
                        return packet.ReadString(packet.ReadUInt16());
                    case VariableLengthSize.FourBytes:
                        return packet.ReadString(packet.ReadInt32());
                }
            else if (attribute.FixedSize > 0)
            {
                return packet.ReadString(attribute.FixedSize);
            }
            throw new InvalidOperationException();
        }

    }

 

Example with MsgAccount packet

Spoiler
    public class MsgAccount
    {
        [PacketField(0)]
        public ushort Length { get; set; }

        [PacketField(1)]
        public ushort Type { get; set; }

        [PacketField(2, FixedSize = 16)]
        public string Username { get; set; }

        [PacketField(3, FixedSize = 16)]
        public string Password { get; set; }

        [PacketField(4, FixedSize = 16)]
        public string ServerName { get; set; }
    }
    class Program
    {
        static void Main(string[] args)
        {
            PacketSerializer<MsgAccount> serializer = new PacketSerializer<MsgAccount>();
            MsgAccount account = new MsgAccount()
            {
                Length = 52,
                Type = 1086, //or maybe not?
                Username = "user",
                Password = "Password",
                ServerName = "Tornado"
            };
            Packet p = new Packet(52);
            serializer.Serialize(account, p);

            Console.WriteLine($"Length: {BitConverter.ToInt16(p.Buffer, 0)}");
            Console.WriteLine($"Type: {BitConverter.ToInt16(p.Buffer, 2)}");
            Console.WriteLine($"Username: {Encoding.ASCII.GetString(p.Buffer, 4, 16)}");
            Console.WriteLine($"Password: {Encoding.ASCII.GetString(p.Buffer, 20, 16)}");
            Console.WriteLine($"Server: {Encoding.ASCII.GetString(p.Buffer, 36, 16)}");

            Console.ReadLine();
        }
    }

 

Output of console is the expected.

Important things to notice:

Packet class is just a BinaryWriter like class, you can see a similar one in a lot of CO related projects

There are errors in SerializationMethods class, check strings for example

Type alias are needed for generic types or arrays, the expression will search for ReadXX where XX is the type name, that is why the StringList alas is used for example.

 

cya!

Edited by 2Explosions
Format

Share this post


Link to post
Share on other sites

Aw dang, I'm bummed this got posted here second to the other forum. I would have responded here first because I trust this board over the other for keeping on topic. Well, If you have any additional questions about what I said there regarding interfaces and generated code (like how Google Protocol Buffers does it), let me know. I like how you're exploring all this. It's all good thoughts and conversation. Keep it up.

Share this post


Link to post
Share on other sites
Posted (edited)

@Spirited Sup! thanks for the interest, i like how nice is to use this board, kinda miss C2PvP tho.

Still exploring alternatives, got a bunch of material to read so i will keep exploring, the code posted is pretty much the prototype that i got working, while it works, it is reaaaally unintuitive to use, after all, it is just a property iterator, and converting it (or maybe using it) for a complete serializer may become a big enough problem to deviate me from my actual project, most likely will code a Conquer specific serializer

As said before (iirc, lazy to read, in other forum 🙄), my current goal is a flexible and scalable project, flexibility as main priority, and is quite a  complex conquer project for me, i will keep itclosed source at least on early stages to avoid influence of more experienced programmers, failing would at least teach me what avoid in future projects. The project is experimental anyways, trying some C# features i haven't used for class implementations, some popular libraries, trying to get used with TDD approach (altough i'm not interested enough to use it during the whole project), also, i'm doing benchmarks and documenting why i'm using X approach, so expect at least a few threads like this one but better structured exploring possible implementations for not so useful things :P

Edited by 2Explosions

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×

Important Information

By using this site, you agree to our Terms of Use.