Out of memory exception in reading a 1.5 GB tile-base Tiff file using LibTiff.Net

Question

I tried to read and write a 1.5 GB tiled Tiff file using LibTiff.Net library as it's declared that support BigTiff (>4 GB) image files. I wrote the code below but get an error in line "buffer[tiles]..." which throws out of memory exception. I would appreciate developers who can help me to solve this problem.

using (Tiff input = Tiff.Open(@"E:\active folder\Sample_04.tif", "r"))
        {
            int width = input.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
            int height = input.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
            int tileWidth = input.GetField(TiffTag.TILEWIDTH)[0].ToInt();
            int tileLentgh = input.GetField(TiffTag.TILELENGTH)[0].ToInt();

            int samplesPerPixel = input.GetField(TiffTag.SAMPLESPERPIXEL)[0].ToInt();
            int bitsPerSample = input.GetField(TiffTag.BITSPERSAMPLE)[0].ToInt();
            int photo = input.GetField(TiffTag.PHOTOMETRIC)[0].ToInt();

            int tiles = 0;
            int tileSize = input.TileSize();
            byte[][] buffer = new byte[tileSize][];


            for (int y = 0; y < height; y += tileLentgh)
            {
                for (int x = 0; x < width; x += tileWidth)
                {
                    buffer[tiles] = new byte[tileSize];
                    input.ReadTile(buffer[tiles], 0, x, y, 0, 0);
                    tiles++;
                }
            }

            // writing
            using (Tiff output = Tiff.Open("output.tif", "w"))
            {

                output.SetField(TiffTag.SAMPLESPERPIXEL, samplesPerPixel);
                output.SetField(TiffTag.IMAGEWIDTH, width );
                output.SetField(TiffTag.IMAGELENGTH, height);
                output.SetField(TiffTag.BITSPERSAMPLE, bitsPerSample);
                output.SetField(TiffTag.ROWSPERSTRIP, output.DefaultStripSize(0));
                output.SetField(TiffTag.PHOTOMETRIC, photo);
                output.SetField(TiffTag.PLANARCONFIG, PlanarConfig.CONTIG);

                int c = 0;

                for (int y = 0; y < height; y += tileLentgh)
                {
                    for (int x = 0; x < width; x += tileWidth)
                    {
                        output.WriteTile(buffer[c], x, y, 0, 0);
                        c++;
                    }
                }
            }
        }
        System.Diagnostics.Process.Start("output.tif");
    }

how much ram does the machine have? how much swap/page-file space is configured/available? how much memory does your process consume when the exception is thrown? — DarkSquirrel42, Apr 18 '17 at 14:02
The machine supports 6 Gb RAM and throws the exception when the memory reaches 84%. I think it uses 20% of the memory for execution. — Nasser Tahani, Apr 18 '17 at 14:06

score 1 · Accepted Answer · edited May 23 '17 at 11:46

The problem is not the library not supporting BigTiff files, the error is thrown when you try to allocate a huge array. The code you wrote tries to allocate the array in the memory of your computer, expecting that there is enough space there to do so and it seems that there is not.

Handling data with sizes comparable to the available memory on the target system always requires extra attention (that's why you can see BigTiff support emphasized in the library's description).

Fortunately for you, this is not a new problem and there are solutions for this: see some answers here or here.

Basically, the idea behind these solutions is to use your hard drive (or other storage device) to store the data and provide an interface for you to swap the neccessary parts to the memory when needed (just like virtual memory).

Thanks, but I had a doubt about my code to read a tiled Tiff because there is no sample code to read using readTile() method. I will take your advice to handle the huge buffer array. Thanks again! — Nasser Tahani, Apr 18 '17 at 14:39

Out of memory exception in reading a 1.5 GB tile-base Tiff file using LibTiff.Net

1 Answers1