6

I'm working on some university project and got stuck with memory issue. I load a bitmap which takes about 1,5GB on HDD with code below:

Bitmap bmp = new Bitmap(pathToFile);

The issue is that the newly created Bitmap object uses about 3,5GB of RAM which is something I can't understand (that's really BIG wrapper :E). I need to get to the pixel array, and the use of Bitmap class is really helpful (I use LockBits() method later, and process the array byte per byte) but in this case it's total blocker. So here is my question:

Is there any easy way to extract the pixel array without lending additional 2gb?

I'm using c# just to extract the needed array, which is later processed in c++ - maybe I can extract all needed data in c++ (but conversion issue appears here - I'm concentrating on 24bgr format)?

PS: I need to keep the whole bitmap in memory so splitting it into parts is no solution.

PS2: Just to clarify some issues: I know the difference between file extension and file format. The loaded file is uncompressed bitmap 3 bytes per pixel of size ~1.42GB (16k x 32k pixels), so why Bitmap object is more than two times bigger? Any decompressing issues and converting into other format aren't taking place.

Rafal Chmiel
  • 61
  • 1
  • 3
  • Possible duplicate: [http://stackoverflow.com/questions/569889/how-do-i-use-large-bitmaps-in-net?rq=1](http://stackoverflow.com/questions/569889/how-do-i-use-large-bitmaps-in-net?rq=1) – Jason Jun 16 '12 at 17:47
  • @Jason: Yes, but that question has no actual answer. Don't know why they've overseen memory mapped files. – Marcel N. Jun 16 '12 at 17:49
  • What encoding does the bitmap use? Is there RLE compression? – Ben Voigt Jun 16 '12 at 18:02
  • bitmap's coding is 24-bit with no RLE compression so the only thing I want is to deserialize the pixel array :( it's size is 16k X 32k pixels so with 3 bytes per one pixel it takes 1.42GB so I really don't understand why Bitmap object created with that file takes additional 2GB of data – Rafal Chmiel Jun 16 '12 at 18:34
  • Ignoring the on-disk size, what are the dimensions and bit depth of the image. This will tell you the minumum requirements to hold the image data in memory (multiply them all together - assuming its not a indexed image - and bit depth a s bytes per pixel). This will then tell you if it's compression etc or memory overhead of Bitmap class that is your issue. – Wolf5370 Jun 16 '12 at 18:53
  • read the answer above ;) – Rafal Chmiel Jun 16 '12 at 19:28
  • I don't understand why you HAVE to have the entire file in memory... – poy Oct 16 '13 at 12:46

3 Answers3

1

Consider using Memory Mapped Files to access your HUGE data :). An example focused on what you need can be found here: http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx It's in managed code but you might as well use it from equivalent native code.

Let me know if you need more details.

Marcel N.
  • 13,726
  • 5
  • 47
  • 72
  • Can you be more specific and show how it can be used in this particular case? – L.B Jun 16 '12 at 17:54
  • Well, you just traverse the bitmap with a memory mapped view, read data in chunks and push it to the C++ part. I think it even may be more efficient to do it all in C++ and process the bitmap data as you traverse the mem mapped file. – Marcel N. Jun 16 '12 at 17:56
  • The whole thing is that with MMF you don't load the whole data in memory. Just a small part of it. And you can walk through it without actually caring that it's not all in memory. – Marcel N. Jun 16 '12 at 18:01
  • I know what MMF is. Data is already on disk as file and i can read it directly from it. No need for MMF here. The problem here is that `Bitmap` class tries to allocate all space in RAM which uses this huge memory. – L.B Jun 16 '12 at 18:07
  • Yes, and the fact that it uses that much memory is an obvious problem. Why use the .NET Bitmap class when you can do a little extra processing and get much more efficiency out of it? I'd take the latter any day. And it's not like you can improve the Bitmap loading. It does what it does and it's reasonable, when you think about it. – Marcel N. Jun 16 '12 at 18:09
  • 1
    `Why use the .NET Bitmap class when you can do a little extra processing and get much more efficiency out of it` this is what OP asks. Explain that part.(Although it still doesn't require MMF) – L.B Jun 16 '12 at 18:13
  • Then no matter how you do it (without MMF) how can you NOT load the entire data in memory? This is what I'm trying to say. I'm offering an alternative to loading all 3GB of data in one go. If you don't understand this, then I don't think you know what MMF does. – Marcel N. Jun 16 '12 at 18:15
  • I say your answer doesn't help OP. Don't get rude. I wrote MMF apps while you were at primary school. – L.B Jun 16 '12 at 18:20
  • 1
    I'm not trying to be rude. Sorry! But I don't know why you say MMF won't work and not only that, why you think there's a better solution. For my part, I think this is the only solution. Looking into how this can be done with Bitmap it's a waste of time. For example, ImageMagick is also using MMF for large files for handling and processing large images (http://www2.fz-juelich.de/vislab/software/www/FAQ.html#C9). – Marcel N. Jun 16 '12 at 18:23
0

You can use this solution , Work with bitmaps faster in C# http://www.codeproject.com/Tips/240428/Work-with-bitmap-faster-with-Csharp

Or you can use memory mapped files

http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx

Reza Farshi
  • 963
  • 7
  • 9
0

You can stop memory caching.

Instead of

Bitmap bmp = new Bitmap(pathToFile);

Use

var bmp = (Bitmap)Image.FromStream(sourceFileStream, false, false);

see https://stackoverflow.com/a/47424918/887092

Kind Contributor
  • 17,547
  • 6
  • 53
  • 70