4

I am getting this error and can not figure why it is happening

I am using Windows 8.1 64 bit on 32 GB ram memory

Application is 32 bit and using only about 400 mb in task manager and even less when checking with GC.GetTotalMemory(false)

How can I debug this error cause ?

C# wpf .net 4.5 application

Here biggest compressed string data in the database (322 KB) : http://www.monstermmorpg.com/reciprocal/zipped.txt

Here uncompressed string of that biggest data (4837 KB) : http://www.monstermmorpg.com/reciprocal/unzipped.txt

Here decompress string function

private static string Un_GZip_String(string compressedText)
{     
        using (var memoryStream = new MemoryStream())
        {
            byte[] gZipBuffer = Convert.FromBase64String(compressedText);
            int dataLength = BitConverter.ToInt32(gZipBuffer, 0);
            memoryStream.Write(gZipBuffer, 4, gZipBuffer.Length - 4);

            var buffer = new byte[dataLength];

            memoryStream.Position = 0;
            using (var gZipStream = new GZipStream(memoryStream, System.IO.Compression.CompressionMode.Decompress))
            {
                gZipStream.Read(buffer, 0, buffer.Length);
            }

            return Encoding.UTF8.GetString(buffer);
        }
}

https://i.stack.imgur.com/fepMs.png

enter image description here

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Furkan Gözükara
  • 22,964
  • 77
  • 205
  • 342
  • Could be a bug in the DecompressString method. Step into it and see what line it actually throws the exception? – Quintium Sep 23 '14 at 21:55
  • Well how big is the string, and how big is the uncompressed version? – Jon Skeet Sep 23 '14 at 21:55
  • @JonSkeet both pretty small. here the string decompressed : http://pastebin.com/AGrabX2s - decompress function is gzip – Furkan Gözükara Sep 23 '14 at 21:56
  • @Quintium added decompress string function – Furkan Gözükara Sep 23 '14 at 21:58
  • Did you try to step into it? – Patrice Gahide Sep 23 '14 at 22:08
  • 1
    You get dataLength from the first 4 bytes of something that comes in in base 64 notation, and use it blindly. Maybe you should be checking for corrupt value? For example it could be a big-endian vs little-endian problem. – RenniePet Sep 23 '14 at 22:10
  • @RenniePet i am not using anything blindly. All data is also composed by my application. Also it was successfully decompressed. I could see text data of hdMyDoc. So it was clearly happened at LoadHtml function – Furkan Gözükara Sep 23 '14 at 22:17
  • 1
    Indeed, step through it and inspect all parameters/variables. Maybe also doing smaller steps in code, with sanity check, and for tracing/debugging. Also, store the Html before loading to a temp file and remove it afterwards to see on what files it crashes. Then try hdMyDoc.LoadHTML on the file. You may have found a bug there. – Pieter21 Sep 23 '14 at 22:19
  • @Pieter21 it happens randomly so there is no way to track it. Also here biggest compressed string data in the database. 329712 character length : http://pastebin.com/wKRwsZ0F – Furkan Gözükara Sep 23 '14 at 22:24
  • @JonSkeet here biggest compressed string in the database : http://pastebin.com/wKRwsZ0F – Furkan Gözükara Sep 23 '14 at 22:25
  • Randomly or not, it's still not clear what caused the OOME (`LoadHtml` or `Un_GZip_String`). Try what Pieter21 suggests (separate the two calls) to see where exactly it crashes. – Patrice Gahide Sep 23 '14 at 22:28
  • @PatriceGahide i just tried in another application and it worked like a charm with : http://img.ctrlv.in/img/14/09/23/5421f580a0c27.png So it is not related to particular case. It is related to overall application ram usage i believe however there is sufficient ram. – Furkan Gözükara Sep 23 '14 at 22:34
  • @PatriceGahide updated link http://img.ctrlv.in/img/14/09/23/5421f580a0c27.png . I tried that thing with biggest compressed string in the database. Also the application does not give any error while running as 64 bit. So all values are correct. – Furkan Gözükara Sep 23 '14 at 22:37
  • Here the biggest compressed string and decompressed versions : Here biggest compressed string data in the database (322 KB) : http://www.monstermmorpg.com/reciprocal/zipped.txt Here uncompressed string of that biggest data (4837 KB) : http://www.monstermmorpg.com/reciprocal/unzipped.txt – Furkan Gözükara Sep 23 '14 at 22:38
  • 6
    The amount of RAM is not the issue here. You stated that this is a 32 bit process, so the *address space* is limited to 2 GB regardless of how much RAM is available on the box. This could be related to fragmentation, but it sounds odd that you would see OOM if you're only using 400 MB at the process level. – Brian Rasmussen Sep 23 '14 at 22:38
  • @BrianRasmussen yes. Also GC.GetTotalMemory(false) shows even less than that. About 300 mb. – Furkan Gözükara Sep 23 '14 at 22:40
  • 2
    It could be large object heap fragmentation. See http://support.microsoft.com/kb/2020006. – phoog Sep 23 '14 at 22:40
  • What does the stack trace say exactly? – Patrice Gahide Sep 23 '14 at 23:02
  • @PatriceGahide it doesn't give info. You see when memory error happens many things get disabled. – Furkan Gözükara Sep 23 '14 at 23:03
  • @PatriceGahide as you can see here i clicked show details and it says evaluation was disabled : http://i.stack.imgur.com/fepMs.png – Furkan Gözükara Sep 23 '14 at 23:08
  • Right, I didn't see that. I guess you're ready for some [low-level memory debugging](http://msdn.microsoft.com/en-us/library/bb190764(v=vs.110).aspx). – Patrice Gahide Sep 23 '14 at 23:25
  • Recommend taking a read of the answers to this question - [How to diagnose an out of memory exception](http://stackoverflow.com/questions/3700490/how-to-diagnose-a-system-outofmemoryexception). – dodgy_coder Sep 24 '14 at 00:58
  • GC.GetTotalMemory method is not giving to you a true picture of what's going on, you need to look at 'task manager'\'performance counter' and see the value 'Working Set(Memory)'. If this is around 1.0 - 1.2 Gb then I am not surprised you're getting OOM exceptions, just becasue it is a 2 Gb address space, doesn't mean you'll get that in the managed .Net world... – AwkwardCoder Sep 24 '14 at 08:45
  • @AwkwardCoder yes however it was about 400 mb when error occurred. – Furkan Gözükara Sep 24 '14 at 20:01
  • sorry, miss read that bit – AwkwardCoder Sep 24 '14 at 21:16

1 Answers1

0

I'd recommend downloading ANTS Performance Profiler:

http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/

It will pin down exactly why you're getting the out of memory exception, and has a 30 day free trial - that should be enough time for you to figure it out.

As someone said in the comments, it may be the large object heap, as that can spiral out of control quite quickly. The performance profiler will quickly tell you what the issue is though.

Callum Bradbury
  • 936
  • 7
  • 14
  • I tried microsoft visual studio performance profiler and even that ones criples. Because there are just too many objects being created and getting disposed of. It is like 10s of thousands per second. – Furkan Gözükara Sep 24 '14 at 20:01