Why is "decimal" data type non-blittable?

Question

GCHandle.Alloc refuses to pin down an array with structs containing the "decimal" data type, but at the same time works fine with "double". What's the reason for that and can I work around it some way?

I know I can instead use unsafe/fixed to get at pointer to the array, but that won't work with generics. :-(

Full sample code to demonstrate the problem. The first Alloc works, but the second fails with a

Object contains non-primitive or non-blittable data.

    public struct X1
    {
        public double X;
    }

    public struct X2
    {
        public decimal X;
    }

Now try this:

        var x1 = new[] {new X1 {X = 42}};
        var handle1 = GCHandle.Alloc(x1, GCHandleType.Pinned); // Works
        var x2 = new[] { new X2 { X = 42 } };
        var handle2 = GCHandle.Alloc(x2, GCHandleType.Pinned); // Fails

Answer 1

  var handle2 = GCHandle.Alloc(x2, GCHandleType.Pinned);

The runtime makes a hard assumption that you are going to call handle.AddrOfPinnedObject(). Surely you are, very little reason to allocated a pinning handle otherwise. That returns an unmanaged pointer, an IntPtr in C#. Distinct from a managed pointer, the kind you get with the fixed keyword.

It furthermore assumes that you are going to pass this pointer to code that cares about the value size and representation. But otherwise being unable to inject a conversion, that code is going to party on the IntPtr directly. This requires the value type to be blittable, a geeky word that means that the bytes in the value can simply be interpreted or copied directly without any conversion and with decent odds that the code that uses the IntPtr is going to be able to recognize the value correctly.

That's a problem with some .NET types, the bool type is notorious for example. Just try this same code with bool instead of decimal and notice you'll get the exact same exception. System.Boolean is a very difficult interop type, there's no dominant standard that describes what it should look like. It is 4 bytes in the C language and the Winapi, 2 bytes in COM Automation, 1 byte in C++ and several other languages. In other words, the odds that the "other code" is going to have a shot at interpreting the 1 byte .NET value are rather slim. The unpredictable size is especially nasty, that throws off all subsequent members.

Much the same with System.Decimal, there is no widely adopted standard that nails down its internal format. Many languages have no support for it at all, notably C and C++ and if you write code in such a language then you need to use a library. Which might use IEEE 754-2008 decimals but that's a johnny-come-lately and suffers from the "too many standards" problem. At the time the CLI spec was written, IEEE 854-1987 standard was around but it was widely ignored. Still a problem today, there are very few processor designs around that support decimals, I only know of PowerPC.

Long story short, you need to create your own blittable type to store decimals. The .NET designers decided to use the COM Automation Currency type to implement System.Decimal, the dominant implementation back then thanks to Visual Basic. This is extremely unlikely to change, way too much code around that takes a dependency on the internal format, making this code the most likely to be compatible and fast:

    public struct X2 {
        private long nativeDecimal;
        public decimal X {
            get { return decimal.FromOACurrency(nativeDecimal); }
            set { nativeDecimal = decimal.ToOACurrency(value); }
        }
    }

You could also consider uint[] and Decimal.Get/SetBits() but I think it is unlikely it will be faster, you'd have to try.

Answer 2

If you love hacks (and only if you love hack) (but note that this should work)

[StructLayout(LayoutKind.Explicit)]
public struct DecimalSplitted
{
    [FieldOffset(0)]
    public uint UInt0;
    [FieldOffset(4)]
    public uint UInt1;
    [FieldOffset(8)]
    public uint UInt2;
    [FieldOffset(12)]
    public uint UInt3;
}

[StructLayout(LayoutKind.Explicit)]
public struct DecimalToUint
{
    [FieldOffset(0)]
    public DecimalSplitted Splitted;
    [FieldOffset(0)]
    public decimal Decimal;
}

[StructLayout(LayoutKind.Explicit)]
public struct StructConverter
{
    [FieldOffset(0)]
    public decimal[] Decimals;

    [FieldOffset(0)]
    public DecimalSplitted[] Splitted;
}

and then:

var decimals = new decimal[] { 1M, 2M, decimal.MaxValue, decimal.MinValue };

DecimalSplitted[] asUints = new StructConverter { Decimals = decimals }.Splitted;

// Works correctly
var h = GCHandle.Alloc(asUints, GCHandleType.Pinned);

// But here we don't need it :-)
h.Free();

for (int i = 0; i < asUints.Length; i++)
{
    DecimalSplitted ds = new DecimalSplitted
    {
        UInt0 = asUints[i].UInt0,
        UInt1 = asUints[i].UInt1,
        UInt2 = asUints[i].UInt2,
        UInt3 = asUints[i].UInt3,
    };

    Console.WriteLine(new DecimalToUint { Splitted = ds }.Decimal);
}

I'm using two quite famous hacks at the same time: using the [StructLayout(LayoutKind.Explicit)] you can superimpose two value types like a C union and you can even superimpose two value type arrays. The last one has a problem: the Length isn't "recalculated", so if you superimpose a byte[] with a long[], if you put there an array of 8 bytes, both fields will show a Length of 8, but clearly if you try to access the long[1] the program will crash. In this case this isn't a problem, because both structures have the same sizeof.

Note that I've used 4xuint, but I could have used 2xulong, or 16xbyte.

Answer 3

Blittable types do not require conversion when they are passed between managed and unmanaged code. MSDN

This is not the case for Decimal. How will other app or whoever is consuming your data will be able to understand Decimal structure? Workaround is to break down decimal into 2 integers, 1 for digits, 1 for decimal base, for example 12.34 into 1234 and 2 meaning (1234 / 10^2).

To properly convert Decimal to binary use GetBits, opposite operations is a bit tricky, this page has example.

Answer 4

This piece of code (refactored from another SO question, which I cannot find now) works just fine with decimal[]. Your problem is that decimal is blittable, but not primitive and a struct containing non-primitive type is not pinnable (GCHandle.Alloc shows an error "Object contains non-primitive or non-blittable data.").

Why is decimal not a primitive type?

/// <summary>
/// Helper class for generic array pointers
/// </summary>
/// <typeparam name="T"></typeparam>
internal class GenericArrayPinner<T> : IDisposable
{
    GCHandle _pinnedArray;
    private T[] _arr;
    public GenericArrayPinner(T[] arr)
    {
        _pinnedArray = GCHandle.Alloc(arr, GCHandleType.Pinned);
        _arr = arr;
    }
    public static implicit operator IntPtr(GenericArrayPinner<T> ap)
    {

        return ap._pinnedArray.AddrOfPinnedObject();
    }

    /// <summary>
    /// Get unmanaged poinetr to the nth element of generic array
    /// </summary>
    /// <param name="n"></param>
    /// <returns></returns>
    public IntPtr GetNthPointer(int n)
    {
        return Marshal.UnsafeAddrOfPinnedArrayElement(this._arr, n);
    }

    public void Dispose()
    {
        _pinnedArray.Free();
        _arr = null;
    }
}