Reading large file using byte[] gives error

Question

Possible Duplicate:
Hash SHA1 large files (over 2gb) in C#

I have a large file in size of terms and it gives me error "Exception of type 'System.OutOfMemoryException' was thrown."

Anyone is having idea or solution for resolve this issue. Please help. Sample code....

 private string GetSha1()
    {
        string filePath = txtFileName.Text;
        byte[] filebytes = System.IO.File.ReadAllBytes(filePath);
        byte[] hashValue;
        SHA1Managed hashString = new SHA1Managed();

        string hex = "";

        hashValue = hashString.ComputeHash(filebytes);
        foreach (byte x in hashValue)
        {
            hex += String.Format("{0:x2}", x);
        }
        return hex;
    }

I am getting exception at line below in above code....

   byte[] filebytes = System.IO.File.ReadAllBytes(filePath);

The filePath is having file which is having size > 500MB.

Is your system running a 64-bit operating system? Is your code built for a 64-bit platform? Does the system have a few GB of memory free? — HABO, Aug 20 '12 at 13:36
@HABO - I would suggest that it doesn't matter in this scenario given that there is no reason to bring the entire file into memory at the same time. — RQDQ, Aug 20 '12 at 14:07

score 14 · Accepted Answer · edited Aug 20 '12 at 13:38

14

Instead of reading the whole file into memory just pass a stream to ComputeHash

using(var file = File.Open(path,FileMode.Open))
{
   hashValue = hashString.ComputeHash(file);
}

edited Aug 20 '12 at 13:38

RQDQ

15,461
2
32
59

answered Aug 20 '12 at 13:36

L.B

114,136
19
178
224

score 2 · Answer 2 · edited Aug 20 '12 at 13:42

2

Well you've pretty much explained what the problem is. You're trying to read a 500MB file straight into a byte[] because you're using ReadAllBytes(). This isn't really suitable for anything except small files.

If you're wanting to compute a hash for a file, just use a stream as the argument:

using (filestream f = File.Open(path,FileMode.Open))
{
    hashValue = hashString.ComputeHash(f);
}

edited Aug 20 '12 at 13:42

Styxxy

7,462
3
40
45

answered Aug 20 '12 at 13:37

PhonicUK

13,486
4
43
62

You are also right, in you case file is in use error never comes. – amit patel Aug 21 '12 at 05:07

score 1 · Answer 3 · answered Aug 20 '12 at 13:37

1

Maybe you should use System.IO.File.Read and just read a chunk of the file into your byte array at a time instead of reading the entire file at once.

See this example on MSDN on using Read.

answered Aug 20 '12 at 13:37

David Hoerster

28,421
8
67
102

Reading large file using byte[] gives error

3 Answers3

Linked