How to write a List of Byte Arrays to a file c#

Question

I have an uploader that I use to split files up and upload them to my sql server. I then download each chunk and create a temporary file. I am trying to write a list of byte arrays(byte[]) into one file to recreate that file. This is because when I try to read the list of byte arrays into one array, I get an OutOfMemory exception. I am just wondering what the best way to do this is. Thanks!

 string path = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
        int currentRowSelection = fUS_FileDataGridView.CurrentCell.RowIndex;
        var totalNumber = fUS_FileDataGridView.Rows[currentRowSelection].Cells[6].Value;
        for (int i = 1; i < 149; i++)
        {
            using (var stream1 = new FileStream(path + @"\" + i + ".zip", FileMode.Open, FileAccess.Read))
            {
                using (var reader = new BinaryReader(stream1))
                {
                    list_.Add(reader.ReadBytes((int)stream1.Length));
                    stream1.Close();
                    stream1.Dispose();
                    reader.Close();
                    reader.Dispose();

                }
            }
        }
        //array_ = list_.SelectMany(a => a).ToArray();

        filePaths_ = @"C:\Users\ATLAS\Desktop\13\fun.zip";
        foreach (byte[] bytes in list_)
        {
            var doc = System.Text.Encoding.Default.GetString(bytes);
            string textToAdd1 = bytes.ToString();
            try
            {
                using (FileStream fs = File.Create(filePaths_))
                using (StreamWriter writer = new StreamWriter(fs, Encoding.Default, 512))
                {
                    writer.Write(textToAdd1);
                    writer.Close();
                    writer.Dispose();
                }
            }
            finally
            {
            }
        }
    }

Update: my question is different from the others I have found because I cannot put my list of byte arrays into a single array to write a file. I am currently only getting a 1 KB file out of my code where I should be getting a 100KB file.

Update 2: The code below makes much more sense but now I am getting a "stream was not writable error"

filePaths_ = @"C:\Users\ATLAS\Desktop\13\fun.zip";
        using (FileStream fs = File.Create(filePaths_))
        for (int i = 0; i < 151; i++)
        {
            using (var stream1 = new FileStream(path + @"\" + i + ".zip", FileMode.Open, FileAccess.Read))
            {
                using (var reader = new BinaryReader(stream1))
                {
                    using (StreamWriter writer = new StreamWriter(fs, Encoding.Default, 512))
                    {
                        writer.Write(reader);
                    }
                }
            }
        }

http://stackoverflow.com/questions/381508/can-a-byte-array-be-written-to-a-file-in-c — Narek Arzumanyan, Aug 04 '16 at 13:42
@NarekArzumanyan nothing on that page will write "List" to a file though will it? I cant put it into a single array because I get an out of memory exception — TWelles1, Aug 04 '16 at 13:45
Possible duplicate of [C# Append byte array to existing file](http://stackoverflow.com/questions/6862368/c-sharp-append-byte-array-to-existing-file) — sr28, Aug 04 '16 at 14:06

Steve · Answer 1 · 2016-08-04T14:09:44.813

If the problem is out of memory then you should think about how to reduce the amount of memory being used.

I don't know what your requirements are but based on the code you provided you could do all the writing inside your first foreach loop. This way you are only loading one file at a time and GC will free up the memory once you are done with each file.

    using (FileStream fs = File.AppendText(filePaths_)) 
    {
        for (int i = 1; i < 149; i++)
        {
            using (var stream1 = new FileStream(path + @"\" + i + ".zip", FileMode.Open, FileAccess.Read))
            {
                using (var reader = new BinaryReader(stream1))
                {
                    //list_.Add(reader.ReadBytes((int)stream1.Length));
                    //Instead of adding that to list, write them to disk here
                    //fs.Write(...)
                    //...


                    stream1.Close();//No need for this, using is going to call it. 
                    stream1.Dispose();//No need for this, using is going to call it. 
                    reader.Close();//No need for this, using is going to call it. 
                    reader.Dispose();//No need for this, using is going to call it. 

                }
            }
        } 
     }

All the files are combined to a single file, the `using (FileStream fs = File.Create(filePaths_))` should be outside of the `for` loop. — Scott Chamberlain, Aug 04 '16 at 13:55
I used the above code and I did not the get out of memory exception but I only got a 1 KB file(should be much bigger). I will look into this a little more(already been staring at this for three days) and get back to you guys. Thanks! — TWelles1, Aug 04 '16 at 14:14
@TWelles1 you are getting 1kb file because you are using bytes[].ToString(). you already have the "real" string value of the byte array in your doc variable which you should be using — Steve, Aug 04 '16 at 15:11

Artavazd Balayan · Answer 2 · 2016-08-04T13:58:03.843

If I right understand what do you want:

// Fill list_
List<byte[]> list_ = null;
// ....

string filePaths_ = @"C:\Users\ATLAS\Desktop\13\fun.zip";
// Create FileStream and BinaryWriter
using (var fs = File.OpenWrite(filePaths_)){
    using (var bw = new BinaryWriter(fs)){
        foreach (var bytes in list_)
            bw.Write(bytes); // Write each byte array to the stream
    }
}

Updated. You can do it in a better way, direct copy one stream to another:

string filePaths_ = @"C:\Users\ATLAS\Desktop\13\fun.zip";
// Result FileStream and BinaryWriter
using (var fs = File.OpenWrite(filePaths_))
{
    for (int i = 1; i < 149; i++)
    {
        using (var stream1 = new FileStream(path + @"\" + i + ".zip", FileMode.Open, FileAccess.Read))
        {
            // Just copy stream1 to fs
            stream1.CopyTo(fs);
        }
    }
}

score 0 · Answer 3 · answered Aug 04 '16 at 14:04

It sounds like what you effectively want to do is append each byte array to the existing file. So I think at the time you want to write each individual array to the file you would simply open the filestream using FileMode.Append (instead of FileMode.Open).

How to write a List of Byte Arrays to a file c#

3 Answers3