Out of memory exception even for a very small task

Question

I have a TCP application where I can request images from a folder on the server from the client. If I request a small folder, it works fine. If its a big folder it will throw an out of memory exception. But then anything after that, even a folder with 1 file will throw the same out of memory exception.

I thought it might have been the thread that is out of memory, so I tried to put it on a separate thread and task but neither worked. Here is the code I'm using:

    public static void Images(string path)
    {
        new Task(() =>
        {
            try
            {
                string root = lookupDirectoryPath("Application data");
                string backupPath = root + @"\Apple Computer\MobileSync\";
                string imagePath = backupPath + path;

                if (Directory.Exists(imagePath))
                {
                    String[] allfiles = Directory.GetFiles(imagePath, "*.*", SearchOption.AllDirectories);
                    List<Image> allImages = new List<Image>();

                    foreach (string file in allfiles)
                    {
                        using (FileStream stream = new FileStream(file, FileMode.Open, FileAccess.Read))
                        {
                            if (IsImage(stream))
                            {
                                allImages.Add(Image.FromFile(file));
                            }
                        }
                    }

                    if (allImages.Count > 0)
                    {
                        byte[] data = imageListToByteArray(allImages);
                        serverSendByteArray(data, 12);
                    }
                    else
                    {
                        serverSendByteArray(Encoding.Default.GetBytes("backup contained no images"), 1);
                    }
                }
                else
                {
                    serverSendByteArray(Encoding.Default.GetBytes("iphone backup folder does not exist"), 1);
                }
            }
            catch (Exception ex)
            {
                if (ex.GetType().IsAssignableFrom(typeof(OutOfMemoryException)))
                {
                    serverSendByteArray(Encoding.Default.GetBytes("Out of memory, could not send iphone images"), 1);
                }
                else
                {
                    serverSendByteArray(Encoding.Default.GetBytes("Unknown error, could not send iphone images"), 1);
                }
            }
        }).Start();
    }

The exception gets thrown at allImages.Add(Image.FromFile(file));

this is the isImage() function:

        public static bool IsImage(Stream stream)
    {
        stream.Seek(0, SeekOrigin.Begin);

        List<string> jpg = new List<string> { "FF", "D8" };
        List<string> bmp = new List<string> { "42", "4D" };
        List<string> gif = new List<string> { "47", "49", "46" };
        List<string> png = new List<string> { "89", "50", "4E", "47", "0D", "0A", "1A", "0A" };
        List<List<string>> imgTypes = new List<List<string>> { jpg, bmp, gif, png };

        List<string> bytesIterated = new List<string>();

        for (int i = 0; i < 8; i++)
        {
            string bit = stream.ReadByte().ToString("X2");
            bytesIterated.Add(bit);

            bool isImage = imgTypes.Any(img => !img.Except(bytesIterated).Any());
            if (isImage)
            {
                return true;
            }
        }

        return false;
    }

Thanks for any help

A profiler is a tool that looks at your program while it's running and tells you what resources are being used, and where. I don't use C#, so I can't suggest one, but a google search should show examples. If you're using an IDE, it may even have one built-in. — Carcigenicate, Apr 03 '17 at 17:57
`Image` implements `IDisposable`, you are nowhere disposing the images resources — Cleptus, Apr 03 '17 at 17:59

score 0 · Answer 1 · edited May 23 '17 at 11:54

0

Image.FromFile seem to cause the error. In the following question it was a corrupeted image file or running out of file handles, Image.FromStream() did it better. Worth a try cause you already have the stream open:

https://stackoverflow.com/a/2216338/7803013

edited May 23 '17 at 11:54

Community

1
1

answered Apr 03 '17 at 18:08

eMpTy

30
4

so like this: "allImages.Add(Image.FromStream(stream));" ?, that gives me a System.Runtime.InteropServices.ExternalException – Rachel Dockter Apr 03 '17 at 18:13
how does the IsImage function look like? Maybe you have to reset your stream to position 0 before reusing it in Image.FromStream – eMpTy Apr 03 '17 at 18:24
hold on ill add it to the question – Rachel Dockter Apr 03 '17 at 18:26

score 0 · Accepted Answer · answered Apr 03 '17 at 18:54

I tried and I can reproduce your problem. It is definitely out of memory and nothing like "It just seems to be" the memory usage increases to about 4 GB and then the error shows up. Console output is just to see what's happening there.

The Image object seems to be not the best way to save the data.

I tried this and got this to work with many many files. Maybe you can change the code to fit your needs:

            String[] allfiles = Directory.GetFiles(imagePath, "*.*", SearchOption.AllDirectories);
            //List<Image> allImages = new List<Image>();
            List<Byte[]> allImagesBytes = new List<Byte[]>();

            foreach (string file in allfiles)
            {
                using (FileStream stream = new FileStream(file, FileMode.Open, FileAccess.Read))
                {
                    if (IsImage(stream))
                    {
                        Console.Clear();
                        Console.Write(allImagesBytes.Count());
                        //allImages.Add(Image.FromStream(stream));
                        //allImages.Add(Image.FromFile(file));
                        allImagesBytes.Add(File.ReadAllBytes(file));

                    }
                }
            }

thanks, the problem would be converting it back into images, if you had an array of bytes with continuous image code, how would you seperate them? I could put a symbol inbetween like a # byte and do it like that but did you have anything in mind? — Rachel Dockter, Apr 03 '17 at 18:58
convert the list/array of byte arrays in json format could be an approach — eMpTy, Apr 03 '17 at 19:04
hmm yeh, well this was the only answer that helped really so ill accept this one, thanks — Rachel Dockter, Apr 03 '17 at 19:07

Cleptus · Answer 3 · 2017-04-03T18:15:27.347

-1

Try changing this

using (FileStream stream = new FileStream(file, FileMode.Open, FileAccess.Read))
{
    if (IsImage(stream))
    {
        allImages.Add(Image.FromFile(file));
    }
}
...
if (allImages.Count > 0)
{
    byte[] data = imageListToByteArray(allImages);
    serverSendByteArray(data, 12);
}

into this:

using (FileStream stream = new FileStream(file, FileMode.Open, FileAccess.Read))
{
    if (IsImage(stream))
    {
        allImages.Add(Image.FromFile(file));
    }
    stream.Close();
}
....
if (allImages.Count > 0)
{
    byte[] data = imageListToByteArray(allImages);
    foreach(Image img in allImages)
    {
        img.Dispose();
    }
    serverSendByteArray(data, 12);
}

edited Apr 03 '17 at 18:15

answered Apr 03 '17 at 18:06

Cleptus

3,446
4
28
34

it doesnt even get that far, it throws the exception on allImages.Add(Image.FromFile(file)); – Rachel Dockter Apr 03 '17 at 18:07
still getting the out of memory exception for a small file after it throws it once – Rachel Dockter Apr 03 '17 at 18:19

Out of memory exception even for a very small task

3 Answers3