2

I have a scenario like this

while ( until toUpload list empty)
{
   create ftp folder (list.item);

   if (upload using ftp (list.item.fileName))
   {
     update Uploaded list that successfully updated;
   }
}

update database using Uploaded list;

Here I used synchronous method to upload file in this loop. And I got a requirement optimize this upload. I found this article How to improve the Performance of FtpWebRequest? and followed instructions they have provided. Therefore I achieved 2000 seconds to 1000 seconds time. Then I move in to upload using asynchronous as in this example in msdn. http://msdn.microsoft.com/en-us/library/system.net.ftpwebrequest(v=vs.90).aspx. I have changed the method 

if (upload using ftp (list.item.fileName))

to 

if (aync upload using ftp (list.item.fileName))

But the things got worst. time become 1000s to 4000s.

I have to upload many file to my ftp server. Therefore best method should be (guessing) asynchronous. Someone help me how to do this in correct manner. I wasn't able to find how to use above msdn example code correctly.

My code - synchronous (exception handling removed to save space):

public bool UploadFtpFile(string folderName, string fileName)
{     
  try
  {
    string absoluteFileName = Path.GetFileName(fileName);           

    var request = WebRequest.Create(new Uri(
      String.Format(@"ftp://{0}/{1}/{2}",
         this.FtpServer, folderName, absoluteFileName))) as FtpWebRequest;
    request.Method = WebRequestMethods.Ftp.UploadFile;
    request.UseBinary = true;
    request.UsePassive = true;
    request.KeepAlive = true;
    request.Credentials = this.GetCredentials();
    request.ConnectionGroupName = this.ConnectionGroupName; 
    request.ServicePoint.ConnectionLimit = 8; 

    using (FileStream fs = File.OpenRead(fileName))
    {
      byte[] buffer = new byte[fs.Length];
      fs.Read(buffer, 0, buffer.Length);
      fs.Close();
      using(var requestStream = request.GetRequestStream())
      {
        requestStream.Write(buffer, 0, buffer.Length);
      }
    }

    using (var response = (FtpWebResponse)request.GetResponse())
    {
        return true;
    }
  }
  catch // real code catches correct exceptions
  {
     return false;  
  }
}

async method

public bool UploadFtpFile(string folderName, string fileName)
{     
  try 
  {
    //this methods is exact method provide in msdn example mail method
    AsyncUploadFtp(String.Format(@"ftp://{0}/{1}/{2}", 
      this.FtpServer, folderName, Path.GetFileName(fileName)), fileName);

    return true;
  }
  catch // real code catches correct exceptions
  {
     return false;  
  }
}
Community
  • 1
  • 1
cdev
  • 5,043
  • 2
  • 33
  • 32
  • 1
    10s means nothing unfortunately... What is the speed you expect (i.e. file size/max upload speed)? Do you know if you can actually send up much more data than you are doing now? (also to get some reasonable advices on your code you'd need to show sample that at least look like C# code). – Alexei Levenkov Feb 14 '13 at 06:31
  • I added my code to question, Do you know if you can actually send up much more data than you are doing now? I reduced 20s to 10s, above example saying he reduced to it 5s using async method. In general environment I'll be able to do the same cause I got first achievement. Please give me some advice. – cdev Feb 14 '13 at 08:17
  • 2
    Advice 1: when posting code make it as small as possible and as useful as possible - i.e. you've posted synchronous upload code for single file and no iteration code nor async version (where most likely issues with async being slower is). Advice 2: measure your max upload speed independently of your code. Advice 3: set your goals - "10s is slow" is not one, something like "up to 1000 files with max combined size 1TB should be uploaded under 3 seconds" could be. Advice 4: profile your code. – Alexei Levenkov Feb 14 '13 at 16:32
  • 1
    How many files are you talking about here? If you've got lots of files, the server may not be able to handle all at once and you might be hitting timeouts/retries or possibly context switching... – Peter Ritchie Feb 14 '13 at 18:37
  • I am calling UploadFtpFile method in a loop. When asyn mood I am calling asynchmethod. So it also in a loop. 10s is the test time. Actual time nearly takes 3+ hours. the link I am using give me upload speed 30kBps for one file (link is 128kBps). In async method this becomes 15 kBps. Forget 10s thing, It s really useless thing. – cdev Feb 15 '13 at 08:44
  • 1) your server may not be able to handle the amount of incoming connections. 2) your internet speed can be a factor as well as the server connection accepting the files. 3) remember, if you are uploading multiple files at the same time, the upload speed is distributed across the files. They all will NOT upload at the same speed. – Ahmed ilyas Mar 11 '14 at 08:48

1 Answers1

0

Have you tried using this in your foreach loop ? 

Task.Factory.StartNew(() => UploadFtpFile(folderName, filename));
JJP
  • 1,439
  • 17
  • 21
  • I have tried something very similar and I got an InnerException saying something about does not support multiple concurrent connections. I don't think this is the solution. – uSeRnAmEhAhAhAhAhA Apr 07 '14 at 06:50