Copy same file from multiple threads to multiple destinations

Question

I am trying to write an application that copies and installer I created over to a list of computers provided. It works perfect with only one computer name in the input but when I add two or more it tells me it cannot read from the source file because its in use.

UPDATE: I am open to other methods of getting this done if this is just a lost cause. My end goal is to simply copy the same file to 500 computers in a datacenter as fast as possible. Then execute the file on each of those computers. Right now for the execution I am using psexec -s so I can bypass UAC.

/*Button Click Code Below*/
/*Install for specific OS*/
foreach (string line in txtServers.Lines)
{
    machine = line;
    lbOutput.Items.Add("Preparing to copy files....");

    /*Copy Files*/
    if(!tWorker.IsAlive)
         tWorker.Start();

    lbOutput.Items.Add("File Copy Complete! Executing installer....");
}

/*File Copy Code Below*/
try
{
   File.Copy("Ionic.Zip.Reduced.dll",@"\\"+machine+@"\C$\Temp\Ionic.Zip.Reduced.dll",true);
   File.Copy("v5.32bit.Install.exe", @"\\" + machine + @"\C$\Temp\v5.32bit.Install.exe", true);
}
catch (Exception ee)
{
   MessageBox.Show(ee.Message);
}

score 4 · Answer 1 · answered Jun 20 '13 at 19:57

4

The magic word is FileShare.Read:

  using(var inputFile = new FileStream(
         "oldFileName.txt", 
         FileMode.Open, 
         FileAccess.Read, 
         FileShare.Read))
     {
        using (var outputFile = new FileStream("newFileName.txt", FileMode.Create))
        { 
            var buffer = new byte[0x10000];
            int bytes;

            while ((bytes = inputFile.Read(buffer, 0, buffer.Length)) > 0) 
            {
                outputFile.Write(buffer, 0, bytes);
            }
        }
    }

Then you should be able to copy it even with multiple threads.

answered Jun 20 '13 at 19:57

Fabian Bigler

10,403
6
47
70

I don't think that using FileShare.ReadWrite is the proper way. If he is copying a file to multiple computers, you shouldn't allow changes until the copy is done. Use FileShare.Read instead. – S Grimminck Jun 20 '13 at 20:00
@SGrimminck: Thanks for the suggestion. I changed it accordingly. – Fabian Bigler Jun 20 '13 at 20:01
Still getting the process cannot access the file "filename" because it is being used by another process. This exception is coming from the try/catch in the second piece of code above with your changes implemented. The exception is showing the remote path. – JD Roberson Jun 20 '13 at 20:09
@JDRoberson do you really have set different output filenames? – Fabian Bigler Jun 20 '13 at 20:26
no, same output name on every computer, its another app I created to install some Microsoft patches and some driver INF's so we can enable SmartCards. – JD Roberson Jun 20 '13 at 20:27
@JDRoberson that is weird. Imo the code should work, I can't help you any further at the moment. Maybe someone else has got an idea. You can also investigate further (Asynchronous File I/O): http://msdn.microsoft.com/en-us/library/kztecsys%28VS.80%29.aspx – Fabian Bigler Jun 20 '13 at 20:39

score 3 · Accepted Answer · answered Jun 20 '13 at 21:26

3

Another way to approach this is to cache the files in memory....

something like

  var filesToLoad = new[] {"a.txt", "b.txt"}.ToList();
  var files = new Dictionary<string, byte[]>();
  filesToLoad.ForEach(f => files.Add(f, File.ReadAllBytes(f)))

then when you want to write them

files.Keys.ToList()
   .ForEach(f => File.WriteAllBytes(Path.Combine(machinePath, f), files[f]));

answered Jun 20 '13 at 21:26

Keith Nicholas

43,549
15
93
156

Definitely the way to go if the files are small enough to keep in memory. – Jim Mischel Jun 20 '13 at 21:43

score 1 · Answer 3 · answered Jun 21 '13 at 12:40

Ok so after an evening of intense frustration it seems I just needed some sleep to figure this out.

I had originally done my foreach in the button click event and since I was starting a thread in that foreach it just continued regardless of the status of everything else. I have since moved that foreach to the thread and the problem was resolved. I can now copy to multiple locations at once and I must say its pretty fast.

Thanks to everyone for their help as always!

score 0 · Answer 4 · answered Jun 20 '13 at 21:43

You might be able to do this by calling CopyFileEx, if you don't specify the COPY_FILE_OPEN_SOURCE_FOR_WRITE flag.

I published a C# wrapper for CopyFileEx a couple of years ago. See A better File.Copy replacement. I don't know if it will do what you want (allow the file to be opened by multiple threads), but it's probably worth a shot.

Copy same file from multiple threads to multiple destinations

4 Answers4

Linked