If I am creating a relatively large structure, how can I calculate the bytes it occupies in memory?
We can do it manually, but if the struct is large enough then how do we do it? Is there some code chunk or application?
If I am creating a relatively large structure, how can I calculate the bytes it occupies in memory?
We can do it manually, but if the struct is large enough then how do we do it? Is there some code chunk or application?
Structs have been troublesome beasts in computer engineering for a very long time. Their memory layout is very hardware dependent. To make them efficient, their members must be aligned so the CPU can read and write their values quickly without having to multiplex the bytes to fit the memory bus width. Every compiler has its own strategy of packing the members, often directed by, for example, the #pragma pack directive in a C or C++ program.
Which is okay, but rather a problem in interop scenarios. Where one chunk of code may make different assumptions about the structure layout than another chunk, compiled by a different compiler. You can see this back in COM, .NET's grandfather solution to interop programming. COM has very poor support for handling structs. It doesn't support them as a native automation type but has a workaround through the IRecordInfo interface. Which lets a program discover the memory layout at runtime through an explicit declaration of the structure in a type library. Which works okay, but is quite inefficient.
The .NET designers made a very courageous, and correct, decision to solve this problem. They made the memory layout of a struct completely undiscoverable. There is no documented way to retrieve the offset of a member. And by extension, no way to discover the size of the struct. Everybody's favorite answer, use Marshal.SizeOf() is not in fact the solution. That returns the size of struct after it is marshaled, the size you'd need to pass to, say, Marshal.AllocCoTaskMem() before you call Marshal.StructureToPtr. That arranges and aligns the struct members according to the [StructLayout] attribute that's associated with the struct. Note that this attribute isn't required for structs (like it is for classes), the runtime implements a default one which uses the declared order for the members.
One very nice side-effect of the layout being undiscoverable is that the CLR can play tricks with it. When packing the members of the struct and aligning them, the layout can get holes that don't store any data. Called padding bytes. Given that the layout is undiscoverable, the CLR can actually use the padding. It moves a member if it is small enough to fit such a hole. You'll now actually get a struct whose size is smaller than what would normally be required given the declared structure layout. And, notably, Marshal.SizeOf() will return the wrong value for the structure size, it returns a value that's too large.
Long story short, there's no general way to get an accurate value for the structure size programmatically. The best thing to do is to just not ask the question. Marshal.SizeOf() will give you a guesstimate, assuming the structure is blittable. If you need an accurate value for some reason then you could look at the generated machine code of a method that declares a local variable of the structure type and compare it to the same method without that local variable. You'll see the difference in the stack pointer adjustment, the "sub esp, xxx" instruction at the top of the method. Of course, it will be architecture dependent, you'll typically get a larger structure in 64-bit mode.
You can use either the sizeof
operator or SizeOf
function.
There are some differences between those options, see the reference links for more.
Anyway a good way to use that function is to have a generic method or extension method like these:
static class Test
{
static void Main()
{
//This will return the memory usage size for type Int32:
int size = SizeOf<Int32>();
//This will return the memory usage size of the variable 'size':
//Both lines are basically equal, the first one makes use of ex. methods
size = size.GetSize();
size = GetSize(size);
}
public static int SizeOf<T>()
{
return System.Runtime.InteropServices.Marshal.SizeOf(typeof(T));
}
public static int GetSize(this object obj)
{
return System.Runtime.InteropServices.Marshal.SizeOf(obj);
}
}
You can either use the sizeof()
keyword for user-defined structs that do not contain any fields or properties that are reference types, or use the Marshal.SizeOf(Type)
or Marshal.SizeOf(object)
to obtain the unmanaged size of a type or a struct that has a sequential or explicit layout.
0. For the example code:
using System;
using System.Diagnostics;
using System.Reflection.Emit;
using System.Runtime.InteropServices;
1. Demonstration struct
[Serializable, StructLayout(LayoutKind.Sequential, Pack = 128)]
public struct T
{
public int a;
public byte b;
public int c;
public String d;
public short e;
};
2. Subtracting managed pointers:
/// Return byte-offset between managed addresses of struct instances 'hi' and 'lo' public static long IL<T1,T2>.RefOffs(ref T1 hi, ref T2 lo) { ... }
public static class IL<T1, T2>
{
public delegate long _ref_offs(ref T1 hi, ref T2 lo);
public static readonly _ref_offs RefOffs;
static IL()
{
var dm = new DynamicMethod(
Guid.NewGuid().ToString(),
typeof(long),
new[] { typeof(T1).MakeByRefType(), typeof(T2).MakeByRefType() },
typeof(Object),
true);
var il = dm.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Sub);
il.Emit(OpCodes.Conv_I8);
il.Emit(OpCodes.Ret);
RefOffs = (_ref_offs)dm.CreateDelegate(typeof(_ref_offs));
}
};
3. Reveal managed internal struct layout:
static class demonstration
{
/// Helper thunk that enables automatic type-inference from argument types
static long RefOffs<T1,T2>(ref T1 hi, ref T2 lo) => IL<T1,T2>.RefOffs(ref hi, ref lo);
public static void Test()
{
var t = default(T);
var rgt = new T[2];
Debug.Print("Marshal.Sizeof<T>: {0,2}", Marshal.SizeOf<T>());
Debug.Print("&rgt[1] - &rgt[0]: {0,2}", RefOffs(ref rgt[1], ref rgt[0]));
Debug.Print("int &t.a {0,2}", RefOffs(ref t.a, ref t));
Debug.Print("byte &t.b {0,2}", RefOffs(ref t.b, ref t));
Debug.Print("int &t.c {0,2}", RefOffs(ref t.c, ref t));
Debug.Print("String &t.d {0,2}", RefOffs(ref t.d, ref t));
Debug.Print("short &t.e {0,2}", RefOffs(ref t.e, ref t));
}
};
4. Results and Discussion
The StructLayout(..., Pack)
setting can be added to the declaration of struct T
with any of the following values: { 0, 1, 2, 4, 8, 16, 32, 64, 128 }. The default value when Pack
is not specified—or equivalently with Pack=0
—sets packing equal to IntPtr.Size
(4
on x86, 8
on x64).
The results of running the above program show that the Pack
value only affects the marshaling size reported by Marshal.SizeOf
, and not the actual size of a single T
memory image, assumed to be the byte-offset between physically adjacent instances. The test code measures this via diagnostic managed array new T[2]
assigned to rgt.
========= x86 ==========
========= x64 ==========
-------- Pack=1 --------
-------- Pack=1 --------
Marshal.Sizeof(): 15
Marshal.Sizeof(): 19
&rgt[1] - &rgt[0]: 16
&rgt[1] - &rgt[0]: 24
-------- Pack=2 --------
-------- Pack=2 --------
Marshal.Sizeof(): 16
Marshal.Sizeof(): 20
&rgt[1] - &rgt[0]: 16
&rgt[1] - &rgt[0]: 24
--- Pack=4/0/default ---
-------- Pack=4 --------
Marshal.Sizeof(): 20
Marshal.Sizeof(): 24
&rgt[1] - &rgt[0]: 16
&rgt[1] - &rgt[0]: 24
-------- Pack=8 --------
--- Pack=8/0/default ---
Marshal.Sizeof(): 20
Marshal.Sizeof(): 32
&rgt[1] - &rgt[0]: 16
&rgt[1] - &rgt[0]: 24
-- Pack=16/32/64/128 ---
-- Pack=16/32/64/128 ---
Marshal.Sizeof(): 20
Marshal.Sizeof(): 32
&rgt[1] - &rgt[0]: 16
&rgt[1] - &rgt[0]: 24
As noted, we find that per-architecture (x86, x64), the managed field layout is consistent regardless of Pack
setting. Here are the actual managed field offsets, again for both 32- and 64-bit mode, as reported by the code above:
┌─offs─┐
field type size x86 x64
===== ====== ==== === ===
a int 4 4 8
b byte 1 14 18
c int 4 8 12
d String 4/8 0 0
e short 2 12 16
The most important thing to notice in this table is that, (as mentioned by Hans), the reported field offsets are non-monotonic with respect to their declaration order. The fields of ValueType
instances are always reordered so that all of the reference-typed fields go first. We can see that the String
field d is at offset 0.
Further reordering optimizes the field order in order to share internal padding excess which would otherwise be wasted. We can see this with byte
field b, which has been moved from being the second declared field, all the way to being the last.
Naturally, by sorting the rows of the previous table we can reveal the true internal managed layout of a .NET ValueType
. Notice that we are able to obtain this layout despite the example struct T
containing a managed reference (String d
) and thus being non-blittable:
============= x86 ============
============= x64 ============
field type size offs end
field type size offs end
===== ====== ==== ==== ===
===== ====== ==== ==== ===
d String 4 0 … 4
d String 8 0 … 8
a int 4 4 … 8
a int 4 8 … 12
c int 4 8 … 12
c int 4 12 … 16
e short 2 12 … 14
e short 2 16 … 18
b byte 1 14 … 15
b byte 1 18 … 19
internal padding: 1 15 … 16
internal padding: 5 19 … 24
x86 managed total size: 16
x64 managed total size: 24
Earlier we determined the size of a single managed struct instance by computing the byte-offset difference between adjacent instances. Taking that into account, the last lines of the previous table trivially show the padding that the CLR internally applies to the end of the example struct T
. Remember again of course, that this internal padding is fixed by the CLR and entirely beyond our control.
5. Coda
For completeness, this last table shows the amount of padding that will be synthesized on-the-fly during marshaling. Notice that in some cases, this Marshal
padding is negative versus the internal managed size. For example, even though the internal managed size of a T
in x64 is 24 bytes, the struct emitted by marshaling can be 19 or 20 bytes with Pack=1
or Pack=2
, respectively.
pack size offs end
pack size offs end
============= ==== ==== ===
============= ==== ==== ===
1 0 15 … 15
1 0 19 … 19
2 1 15 … 16
2 1 19 … 20
4/8/16/32/64… 5 15 … 20
4/8/16/32/64… 5 19 … 24
I wrote a tiny little library in CIL (.NET's assembly language) to expose some neat functionality that isn't available in C#. I broke out the sizeof
instruction.
It differs significantly from the sizeof
operator in C#. Basically, it gets the size of a structure (or reference type, which act funny with some optimizations) including padding and all. So, if you were to create an array of T
, then you can use sizeof to determine the distance between each array element in bytes. It's also completely verifiable and managed code. Note that in Mono there was a bug (pre-3.0?) which would cause the sizeof reference types to be reported incorrectly, which would expand to structures containing reference types.
Anyway, you can download the BSD licensed library (and CIL) from BitBucket. You can also see some example code and a few more details at my blog.
In .NET Core, the sizeof
CIL instruction has been exposed via the recently added Unsafe
class. Add a reference to the System.Runtime.CompilerServices.Unsafe
package and just do this:
int size = Unsafe.SizeOf<MyStruct>();
It works for reference types as well (will return 4 or 8 depending on your computer's architecture).
You want to use System.Runtime.InteropServices.Marshal.SizeOf():
struct s
{
public Int64 i;
}
public static void Main()
{
s s1;
s1.i = 10;
var s = System.Runtime.InteropServices.Marshal.SizeOf(s1);
}
You can use System.Runtime.InteropServices.Marshal.SizeOf()
to get the size in bytes as well.