Bit fields in C#

Question

I have a structure which I need to populate and write to disk (several actually).

An example is:

byte-6    
bit0 - original_or_copy  
bit1 - copyright  
bit2 - data_alignment_indicator  
bit3 - PES_priority  
bit4-bit5 - PES_scrambling control.  
bit6-bit7 - reserved  

In C I might do something like the following:

struct PESHeader  {
    unsigned reserved:2;
    unsigned scrambling_control:2;
    unsigned priority:1;
    unsigned data_alignment_indicator:1;
    unsigned copyright:1;
    unsigned original_or_copy:1;
};

Is there any way to do this in C# that would enable me to access the bits using the struct dereferencing dot operator?

For a couple of structures, I can just do bit shifting wrapped in an accessor function.

I have loads of structures to handle in this way, so I'm looking for something that's easier to read and quicker to write.

Answer 1

I'd probably knock together something using attributes, then a conversion class to convert suitably attributed structures to the bitfield primitives. Something like...

using System;

namespace BitfieldTest
{
    [global::System.AttributeUsage(AttributeTargets.Field, AllowMultiple = false)]
    sealed class BitfieldLengthAttribute : Attribute
    {
        uint length;

        public BitfieldLengthAttribute(uint length)
        {
            this.length = length;
        }

        public uint Length { get { return length; } }
    }

    static class PrimitiveConversion
    {
        public static long ToLong<T>(T t) where T : struct
        {
            long r = 0;
            int offset = 0;

            // For every field suitably attributed with a BitfieldLength
            foreach (System.Reflection.FieldInfo f in t.GetType().GetFields())
            {
                object[] attrs = f.GetCustomAttributes(typeof(BitfieldLengthAttribute), false);
                if (attrs.Length == 1)
                {
                    uint fieldLength  = ((BitfieldLengthAttribute)attrs[0]).Length;

                    // Calculate a bitmask of the desired length
                    long mask = 0;
                    for (int i = 0; i < fieldLength; i++)
                        mask |= 1 << i;

                    r |= ((UInt32)f.GetValue(t) & mask) << offset;

                    offset += (int)fieldLength;
                }
            }

            return r;
        }
    }

    struct PESHeader
    {
        [BitfieldLength(2)]
        public uint reserved;
        [BitfieldLength(2)]
        public uint scrambling_control;
        [BitfieldLength(1)]
        public uint priority;
        [BitfieldLength(1)]
        public uint data_alignment_indicator;
        [BitfieldLength(1)]
        public uint copyright;
        [BitfieldLength(1)]
        public uint original_or_copy;
    };

    public class MainClass
    {
        public static void Main(string[] args)
        {
            PESHeader p = new PESHeader();

            p.reserved = 3;
            p.scrambling_control = 2;
            p.data_alignment_indicator = 1;

            long l = PrimitiveConversion.ToLong(p);


            for (int i = 63; i >= 0; i--)
            {
                Console.Write( ((l & (1l << i)) > 0) ? "1" : "0");
            }

            Console.WriteLine();

            return;
        }
    }
}

Which produces the expected ...000101011. Of course, it needs more error checking and a slightly saner typing, but the concept is (I think) sound, reusable, and lets you knock out easily maintained structures by the dozen.

adamw

Answer 2

By using an enum you can do this, but will look awkward.

[Flags]
public enum PESHeaderFlags
{
    IsCopy = 1, // implied that if not present, then it is an original
    IsCopyrighted = 2,
    IsDataAligned = 4,
    Priority = 8,
    ScramblingControlType1 = 0,
    ScramblingControlType2 = 16,
    ScramblingControlType3 = 32,
    ScramblingControlType4 = 16+32,
    ScramblingControlFlags = ScramblingControlType1 | ScramblingControlType2 | ... ype4
    etc.
}

Answer 3

You want StructLayoutAttribute

[StructLayout(LayoutKind.Explicit, Size=1, CharSet=CharSet.Ansi)]
public struct Foo 
{ [FieldOffset(0)]public byte original_or_copy; 
  [FieldOffset(0)]public byte copyright;
  [FieldOffset(0)]public byte data_alignment_indicator; 
  [FieldOffset(0)]public byte PES_priority; 
  [FieldOffset(0)]public byte PES_scrambling_control; 
  [FieldOffset(0)]public byte reserved; 
}

This is really a union but you can use it as a bitfield--you just have to be conscious of where in the byte the bits for each field are supposed to be. Utility functions and/or constants to AND against can help.

const byte _original_or_copy = 1;
const byte _copyright        = 2;

//bool ooo = foo.original_or_copy();
static bool original_or_copy(this Foo foo) 
{ return  (foo.original_or_copy & _original_or_copy)  == original_or_copy;
}    

There is also LayoutKind.Sequential which will allow you to do it the C way.

Answer 4

As Christophe Lambrechts suggested BitVector32 provides a solution. Jitted performance should be adequate, but don't know for sure. Here's the code illustrating this solution:

public struct rcSpan
{
    //C# Spec 10.4.5.1: The static field variable initializers of a class correspond to a sequence of assignments that are executed in the textual order in which they appear in the class declaration.
    internal static readonly BitVector32.Section sminSection = BitVector32.CreateSection(0x1FFF);
    internal static readonly BitVector32.Section smaxSection = BitVector32.CreateSection(0x1FFF, sminSection);
    internal static readonly BitVector32.Section areaSection = BitVector32.CreateSection(0x3F, smaxSection);

    internal BitVector32 data;

    //public uint smin : 13; 
    public uint smin
    {
        get { return (uint)data[sminSection]; }
        set { data[sminSection] = (int)value; }
    }

    //public uint smax : 13; 
    public uint smax
    {
        get { return (uint)data[smaxSection]; }
        set { data[smaxSection] = (int)value; }
    }

    //public uint area : 6; 
    public uint area
    {
        get { return (uint)data[areaSection]; }
        set { data[areaSection] = (int)value; }
    }
}

You can do a lot this way. You can do even better without using BitVector32, by providing handmade accessors for every field:

public struct rcSpan2
{
    internal uint data;

    //public uint smin : 13; 
    public uint smin
    {
        get { return data & 0x1FFF; }
        set { data = (data & ~0x1FFFu ) | (value & 0x1FFF); }
    }

    //public uint smax : 13; 
    public uint smax
    {
        get { return (data >> 13) & 0x1FFF; }
        set { data = (data & ~(0x1FFFu << 13)) | (value & 0x1FFF) << 13; }
    }

    //public uint area : 6; 
    public uint area
    {
        get { return (data >> 26) & 0x3F; }
        set { data = (data & ~(0x3F << 26)) | (value & 0x3F) << 26; }
    }
}

Surprisingly this last, handmade solution seems to be the most convenient, least convoluted, and the shortest one. That's of course only my personal preference.

Answer 5

One more based off of Zbyl's answer. This one is a little easier to change around for me - I just have to adjust the sz0,sz1... and also make sure mask# and loc# are correct in the Set/Get blocks.

Performance wise, it should be the same as they both resolved to 38 MSIL statements. (constants are resolved at compile time)

public struct MyStruct
{
    internal uint raw;

    const int sz0 = 4, loc0 = 0,          mask0 = ((1 << sz0) - 1) << loc0;
    const int sz1 = 4, loc1 = loc0 + sz0, mask1 = ((1 << sz1) - 1) << loc1;
    const int sz2 = 4, loc2 = loc1 + sz1, mask2 = ((1 << sz2) - 1) << loc2;
    const int sz3 = 4, loc3 = loc2 + sz2, mask3 = ((1 << sz3) - 1) << loc3;

    public uint Item0
    {
        get { return (uint)(raw & mask0) >> loc0; }
        set { raw = (uint)(raw & ~mask0 | (value << loc0) & mask0); }
    }

    public uint Item1
    {
        get { return (uint)(raw & mask1) >> loc1; }
        set { raw = (uint)(raw & ~mask1 | (value << loc1) & mask1); }
    }

    public uint Item2
    {
        get { return (uint)(raw & mask2) >> loc2; }
        set { raw = (uint)(raw & ~mask2 | (value << loc2) & mask2); }
    }

    public uint Item3
    {
        get { return (uint)((raw & mask3) >> loc3); }
        set { raw = (uint)(raw & ~mask3 | (value << loc3) & mask3); }
    }
}

Answer 6

You could also use the BitVector32 and especially the Section struct. The example is very good.

Answer 7

I find myself quite comfortable with these helper functions:

uint SetBits(uint word, uint value, int pos, int size)
{
    uint mask = ((((uint)1) << size) - 1) << pos;
    word &= ~mask; //resettiamo le posizioni
    word |= (value << pos) & mask;
    return word;
}

uint ReadBits(uint word, int pos, int size)
{
    uint mask = ((((uint)1) << size) - 1) << pos;
    return (word & mask) >> pos;
}

then:

uint the_word;

public uint Itemx
{
    get { return ReadBits(the_word, 5, 2); }
    set { the_word = SetBits(the_word, value, 5, 2) }
}

Answer 8

While it is a class, using BitArray seems like the way to least reinvent the wheel. Unless you're really pressed for performance, this is the simplest option. (Indexes can be referenced with the [] operator.)

Answer 9

Could an Enum with the Flags Attribute help maybe? See here:

What does the [Flags] Enum Attribute mean in C#?

Answer 10

A flags enum can work too, I think, if you make it a byte enum:

[Flags] enum PesHeaders : byte { /* ... */ }

Answer 11

I wrote one, share it, may help someone:

[global::System.AttributeUsage(AttributeTargets.Field, AllowMultiple = false)]
public sealed class BitInfoAttribute : Attribute {
    byte length;
    public BitInfoAttribute(byte length) {
        this.length = length;
    }
    public byte Length { get { return length; } }
}

public abstract class BitField {

    public void parse<T>(T[] vals) {
        analysis().parse(this, ArrayConverter.convert<T, uint>(vals));
    }

    public byte[] toArray() {
        return ArrayConverter.convert<uint, byte>(analysis().toArray(this));
    }

    public T[] toArray<T>() {
        return ArrayConverter.convert<uint, T>(analysis().toArray(this));
    }

    static Dictionary<Type, BitTypeInfo> bitInfoMap = new Dictionary<Type, BitTypeInfo>();
    private BitTypeInfo analysis() {
        Type type = this.GetType();
        if (!bitInfoMap.ContainsKey(type)) {
            List<BitInfo> infos = new List<BitInfo>();

            byte dataIdx = 0, offset = 0;
            foreach (System.Reflection.FieldInfo f in type.GetFields()) {
                object[] attrs = f.GetCustomAttributes(typeof(BitInfoAttribute), false);
                if (attrs.Length == 1) {
                    byte bitLen = ((BitInfoAttribute)attrs[0]).Length;
                    if (offset + bitLen > 32) {
                        dataIdx++;
                        offset = 0;
                    }
                    infos.Add(new BitInfo(f, bitLen, dataIdx, offset));
                    offset += bitLen;
                }
            }
            bitInfoMap.Add(type, new BitTypeInfo(dataIdx + 1, infos.ToArray()));
        }
        return bitInfoMap[type];
    }
}

class BitTypeInfo {
    public int dataLen { get; private set; }
    public BitInfo[] bitInfos { get; private set; }

    public BitTypeInfo(int _dataLen, BitInfo[] _bitInfos) {
        dataLen = _dataLen;
        bitInfos = _bitInfos;
    }

    public uint[] toArray<T>(T obj) {
        uint[] datas = new uint[dataLen];
        foreach (BitInfo bif in bitInfos) {
            bif.encode(obj, datas);
        }
        return datas;
    }

    public void parse<T>(T obj, uint[] vals) {
        foreach (BitInfo bif in bitInfos) {
            bif.decode(obj, vals);
        }
    }
}

class BitInfo {

    private System.Reflection.FieldInfo field;
    private uint mask;
    private byte idx, offset, shiftA, shiftB;
    private bool isUnsigned = false;

    public BitInfo(System.Reflection.FieldInfo _field, byte _bitLen, byte _idx, byte _offset) {
        field = _field;
        mask = (uint)(((1 << _bitLen) - 1) << _offset);
        idx = _idx;
        offset = _offset;
        shiftA = (byte)(32 - _offset - _bitLen);
        shiftB = (byte)(32 - _bitLen);

        if (_field.FieldType == typeof(bool)
            || _field.FieldType == typeof(byte)
            || _field.FieldType == typeof(char)
            || _field.FieldType == typeof(uint)
            || _field.FieldType == typeof(ulong)
            || _field.FieldType == typeof(ushort)) {
            isUnsigned = true;
        }
    }

    public void encode(Object obj, uint[] datas) {
        if (isUnsigned) {
            uint val = (uint)Convert.ChangeType(field.GetValue(obj), typeof(uint));
            datas[idx] |= ((uint)(val << offset) & mask);
        } else {
            int val = (int)Convert.ChangeType(field.GetValue(obj), typeof(int));
            datas[idx] |= ((uint)(val << offset) & mask);
        }
    }

    public void decode(Object obj, uint[] datas) {
        if (isUnsigned) {
            field.SetValue(obj, Convert.ChangeType((((uint)(datas[idx] & mask)) << shiftA) >> shiftB, field.FieldType));
        } else {
            field.SetValue(obj, Convert.ChangeType((((int)(datas[idx] & mask)) << shiftA) >> shiftB, field.FieldType));
        }
    }
}

public class ArrayConverter {
    public static T[] convert<T>(uint[] val) {
        return convert<uint, T>(val);
    }

    public static T1[] convert<T0, T1>(T0[] val) {
        T1[] rt = null;
        // type is same or length is same
        // refer to http://stackoverflow.com/questions/25759878/convert-byte-to-sbyte
        if (typeof(T0) == typeof(T1)) { 
            rt = (T1[])(Array)val;
        } else {
            int len = Buffer.ByteLength(val);
            int w = typeWidth<T1>();
            if (w == 1) { // bool
                rt = new T1[len * 8];
            } else if (w == 8) {
                rt = new T1[len];
            } else { // w > 8
                int nn = w / 8;
                int len2 = (len / nn) + ((len % nn) > 0 ? 1 : 0);
                rt = new T1[len2];
            }

            Buffer.BlockCopy(val, 0, rt, 0, len);
        }
        return rt;
    }

    public static string toBinary<T>(T[] vals) {
        StringBuilder sb = new StringBuilder();
        int width = typeWidth<T>();
        int len = Buffer.ByteLength(vals);
        for (int i = len-1; i >=0; i--) {
            sb.Append(Convert.ToString(Buffer.GetByte(vals, i), 2).PadLeft(8, '0')).Append(" ");
        }
        return sb.ToString();
    }

    private static int typeWidth<T>() {
        int rt = 0;
        if (typeof(T) == typeof(bool)) { // x
            rt = 1;
        } else if (typeof(T) == typeof(byte)) { // x
            rt = 8;
        } else if (typeof(T) == typeof(sbyte)) {
            rt = 8;
        } else if (typeof(T) == typeof(ushort)) { // x
            rt = 16;
        } else if (typeof(T) == typeof(short)) {
            rt = 16;
        } else if (typeof(T) == typeof(char)) {
            rt = 16;
        } else if (typeof(T) == typeof(uint)) { // x
            rt = 32;
        } else if (typeof(T) == typeof(int)) {
            rt = 32;
        } else if (typeof(T) == typeof(float)) {
            rt = 32;
        } else if (typeof(T) == typeof(ulong)) { // x
            rt = 64;
        } else if (typeof(T) == typeof(long)) {
            rt = 64;
        } else if (typeof(T) == typeof(double)) {
            rt = 64;
        } else {
            throw new Exception("Unsupport type : " + typeof(T).Name);
        }
        return rt;
    }
}

and the usage:

class MyTest01 : BitField {
    [BitInfo(3)]
    public bool d0;
    [BitInfo(3)]
    public short d1;
    [BitInfo(3)]
    public int d2;
    [BitInfo(3)]
    public int d3;
    [BitInfo(3)]
    public int d4;
    [BitInfo(3)]
    public int d5;

    public MyTest01(bool _d0, short _d1, int _d2, int _d3, int _d4, int _d5) {
        d0 = _d0;
        d1 = _d1;
        d2 = _d2;
        d3 = _d3;
        d4 = _d4;
        d5 = _d5;
    }

    public MyTest01(byte[] datas) {
        parse(datas);
    }

    public new string ToString() {
        return string.Format("d0: {0}, d1: {1}, d2: {2}, d3: {3}, d4: {4}, d5: {5} \r\nbinary => {6}",
            d0, d1, d2, d3, d4, d5, ArrayConverter.toBinary(toArray()));
    }
};

class MyTest02 : BitField {
    [BitInfo(5)]
    public bool val0;
    [BitInfo(5)]
    public byte val1;
    [BitInfo(15)]
    public uint val2;
    [BitInfo(15)]
    public float val3;
    [BitInfo(15)]
    public int val4;
    [BitInfo(15)]
    public int val5;
    [BitInfo(15)]
    public int val6;

    public MyTest02(bool v0, byte v1, uint v2, float v3, int v4, int v5, int v6) {
        val0 = v0;
        val1 = v1;
        val2 = v2;
        val3 = v3;
        val4 = v4;
        val5 = v5;
        val6 = v6;
    }

    public MyTest02(byte[] datas) {
        parse(datas);
    }

    public new string ToString() {
        return string.Format("val0: {0}, val1: {1}, val2: {2}, val3: {3}, val4: {4}, val5: {5}, val6: {6}\r\nbinary => {7}",
            val0, val1, val2, val3, val4, val5, val6, ArrayConverter.toBinary(toArray()));
    }
}

public class MainClass {

    public static void Main(string[] args) {
        MyTest01 p = new MyTest01(false, 1, 2, 3, -1, -2);
        Debug.Log("P:: " + p.ToString());
        MyTest01 p2 = new MyTest01(p.toArray());
        Debug.Log("P2:: " + p2.ToString());

        MyTest02 t = new MyTest02(true, 1, 12, -1.3f, 4, -5, 100);
        Debug.Log("t:: " + t.ToString());
        MyTest02 t2 = new MyTest02(t.toArray());
        Debug.Log("t:: " + t.ToString());

        Console.Read();
        return;
    }
}

Answer 12

I wrote one this morning with T4. :) Same example as Zbyl, though I threw in a bit of uint sizing fun. This is just a first pass, it could obviously use a little error checking. Also the bitFields spec array would be nicer in a separate file, maybe a .ttinclude, or a json/yaml..

=== BitFields.tt ===

<#@ template language="C#" #>
<#@ assembly name="System.Core" #>
<#@ import namespace="System.Linq" #>

<#
var bitFields = new[]
{
    new
    {
        Name = "rcSpan2", Fields = new[] { ("smin", 13), ("smax", 13), ("area", 6) },
    },  
};

foreach (var bitField in bitFields)
{
    static string getType(int size) =>
        size switch
        {
            > 32 => "ulong",
            > 16 => "uint",
            > 8 => "ushort",
            _ => "byte",
        };

    var bitFieldType = getType(bitField.Fields.Sum(f => f.Item2)); 
#>
public struct <#=bitField.Name#>
{
    <#=bitFieldType#> _bitfield;

<#
var offset = 0;
foreach (var (fieldName, fieldSize) in bitField.Fields)
{
    var fieldType = getType(fieldSize);
    var fieldMask = $"0x{((1UL<<fieldSize)-1):X}U";
#>
    public <#=fieldType#> <#=fieldName#> // : <#=fieldSize#>
    {
        get => (<#=fieldType#>)(<#=offset > 0 ? $"(_bitfield >> {offset})" : "_bitfield"#> & <#=fieldMask#>);
        set => _bitfield = (<#=bitFieldType#>)((_bitfield & ~((<#=bitFieldType#>)<#=fieldMask#> << <#=offset#>)) | ((<#=bitFieldType#>)(value & <#=fieldMask#>) << <#=offset#>));
    }  
<#
    offset += fieldSize;
}
#>
}

<#}#>

=== BitFields.cs === (generated)

public struct rcSpan2
{
    uint _bitfield;

    public ushort smin // : 13
    {
        get => (ushort)(_bitfield & 0x1FFFU);
        set => _bitfield = (uint)((_bitfield & ~((uint)0x1FFFU << 0)) | ((uint)(value & 0x1FFFU) << 0));
    }  
    public ushort smax // : 13
    {
        get => (ushort)((_bitfield >> 13) & 0x1FFFU);
        set => _bitfield = (uint)((_bitfield & ~((uint)0x1FFFU << 13)) | ((uint)(value & 0x1FFFU) << 13));
    }  
    public byte area // : 6
    {
        get => (byte)((_bitfield >> 26) & 0x3FU);
        set => _bitfield = (uint)((_bitfield & ~((uint)0x3FU << 26)) | ((uint)(value & 0x3FU) << 26));
    }  
}