13

FileStream's read/write method can take only integer value as length. But FileStreamobject returns length in long. In this case, what if file size is larger than integer value (approximate more than 2GB). Then how FileStream's read/write method handle long value.

Amit
  • 21,570
  • 27
  • 74
  • 94

3 Answers3

13

Then you read and write in multiple chunks. The CLR has a limit on the size of any particular object anyway (also around 2GB IIRC, even on a 64-bit CLR), so you couldn't have a byte array big enough for it to be a problem.

You should always loop when reading anyway, as you can't guarantee that a Read call will read as many bytes as you requested, even if there's more data to come.

EDIT: Reading in chunks:

byte[] buffer = new byte[1024 * 32];
int bytesRead;

while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
    // Use the data you've read
}

Writing in chunks will depend on what you're writing... it's hard to talk about it in the abstract.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Dear, please provide example for Read and Write in chunks. – Amit Apr 13 '11 at 19:22
  • @jams: Have edited my answer, but it's hard to give concrete code without knowing what you're doing with the data. – Jon Skeet Apr 13 '11 at 19:35
  • Actually I have to read and write a video file from HDD, who’s size is bigger than 2GB. I am facing difficulty to understand your example for chunks read/write. But it is not an issue I will manage it. – Amit Apr 13 '11 at 19:59
  • 3
    Dear @JonSkeet you are a gem. When I request you to provide example, I was expecting that you will not provide it, because most senior members on SO behave rudely to such silly requests. But you provided it because you have a nice heart and you are a true gentlemen. I wish you to complete 300K reputation points very soon. – Amit Apr 13 '11 at 20:01
4

There is no need to directly write more than 2Gb of data in one call

If you would really have that amount buffered contiguously in memory (? maby as an unsafely acquired UnmanagedMemoryStream to implement a core dump?) you could easily batch the writes in multiple calls. It will get written to disk in blocks of 512k to max 4k on current hardware, anyway.

The great value of 'streaming' interfaces is that you can have it anywhich way. In fact, when you look into to it you will find that the CLR Arrays (and anything else) are actually bounded to 2GB

Update

Since you have now confessed that you basically want to copy streams, you might be better served with an instant solution. There is File.Copy

File.Copy("file-a.txt", "file-new.txt");

Or there is the standard answer

Stream input
input.CopyTo(output); // .NET 4.0

// .NET 3.5 and others
public static void CopyStream(Stream input, Stream output)
{
    byte[] buffer = new byte[32768];
    while (true)
    {
        int read = input.Read (buffer, 0, buffer.Length);
        if (read <= 0)
            return;
        output.Write (buffer, 0, read);
    }
}

Don't forget about Flushing, Closing and Disposing your Streams as appropriate if you are handling the streams manually.

Cheers

Community
  • 1
  • 1
sehe
  • 374,641
  • 47
  • 450
  • 633
  • Thrown in a handful of helpful hints in reply to OP comment "_Actually I have to read and write a video file from HDD, who’s size is bigger than 2GB_" – sehe Apr 13 '11 at 21:17
0

as far as i know you can still use seek to get to the right position in the stream.

you will probably have to loop to do so, and if you want to read more than 2 gigs wou will need to loop here too

DarkSquirrel42
  • 10,167
  • 3
  • 20
  • 31