OutOfMemoryException with gcAllowVeryLargeObjects

Question

I'm using a BinarySerializer with a pretty big (althought not very deep) graph of items. I have 8GB of ram backed by 12Gig of swap and i'm getting an OutOfMemoryException when serializing which is expected ( it's possible the graph could go near or over 2Gb).

However when i use gcAllowVeryLargeObjects it's no better, i still get the same exception and i'm definately working on something that should hold in memory (at least with the swap).

Is there anything i could do to support serializing this / a way to get the same feature set but getting the result in chuncks maybe?

There's nothing special about my serialization code :

    public static byte[] Serialize(this object o)
    {
        var ms = new MemoryStream();
        var bf = new BinaryFormatter();
        bf.Serialize(ms, o);
        ms.Position = 0;
        return ms.ToArray();
    }

The object i'm serializing contains arrays of items that themselves contains array etc, but the full graph itself isn't "that" large (it's the result of indexing data that, at the source, is already only around 1GB in size).

It's not due to GC fragmentation either (compacting the large heap didn't help).

Is there a reason you're serializing something that big in memory rather than file? If I'm looking at this right, your memory requirements are at least 3 times the size of the structure (structure itself, MemoryStream contents of it, and byte array from MemoryStream) at a single point of execution. Can you serialize to file instead? — LB2, Apr 21 '14 at 21:45
@AlexeiLevenkov AnyCPU on a 64 bit OS (so should be 64 bit unless i missunderstood?) — Ronan Thibaudau, Apr 21 '14 at 21:45
@LB2 It's never getting stored, it's serialized, passed to another process, and deserialized there all at once, i'm using binary serialization for speed mostly — Ronan Thibaudau, Apr 21 '14 at 21:46
Does your process show as "yourprocess.exe *32"? in Task Manager? If so, it is running in 32-bit mode. — Vlad, Apr 21 '14 at 21:50
Damn found it, when did that happen? Haven't seen that check before (and why would someone target any cpu with prefer 32 bit instead of targeting X86?!!) — Ronan Thibaudau, Apr 21 '14 at 21:50
That was it btw, now it's actually running as 64 bit and no longer posing this problem, this has to be the worse opt-out type of change i've ever seen lol. Can you rephrase that as an answer so i can accept it? — Ronan Thibaudau, Apr 21 '14 at 21:53
B/w, you're missing `using` around your `MemoryStream` which may be contributing to your memory woes (though not the core issue here). — LB2, Apr 21 '14 at 21:54
@LB2 Good catch, no clue how i let it slide (what's ironic is i do have it in the deserialization code) — Ronan Thibaudau, Apr 21 '14 at 21:55

score 18 · Accepted Answer · edited May 23 '17 at 12:16

By default AnyCPU runs as 32 bit process on both x86 and x64 OS. So even with gcAllowVeryLargeObjects set on x64 OS you run into 4GB limit of address space (2GB on x86).

To change uncheck "prefer 32 bit" property on solution properties -> "build" tab.

The details and history can be found in following answer: What is the purpose of the "Prefer 32-bit" setting in Visual Studio 2012 and how does it actually work?

OutOfMemoryException with gcAllowVeryLargeObjects

1 Answers1

Linked