0

I have written a console app to download files from an FTP and then upload to a different FTP location. The downloading on the files takes around 10 seconds, but the upload around 6 minutes. There are 256 files each around 5-30KB in size. So very small.

The upload and download code is very similar, it iterates through all files in the directory then uploads. It is fairly simple as seen below, it iterates and uploads files to the ftp from the D:\LEV\ folder.

EDIT: This is run on a Azure 'small' Windows virtual machine, so I assume bandwidth isn't a problem? Also I am performing the same task on another virtual machine using the windows ftp.exe to upload and it is 2 times quicker than my console app on the same machine.

Any clues why it is so slow, or are there ways to improve the speed?

static public void Upload(string file1)
{        
    string upftpServerIP = "ftp://ftp.domain.co.uk/lev/";
    string upftpUserID = "username";
    string upftpPassword = "password";

    string uri = upftpServerIP + file1;
    Uri serverUri = new Uri(uri);
    if (serverUri.Scheme != Uri.UriSchemeFtp)
    {
       return;
    }
    FtpWebRequest reqFTP;
    reqFTP = (FtpWebRequest)FtpWebRequest.Create(new Uri(upftpServerIP + file1));
    reqFTP.Credentials = new NetworkCredential(upftpUserID, upftpPassword);
    reqFTP.KeepAlive = false;
    reqFTP.Method = WebRequestMethods.Ftp.UploadFile;
    reqFTP.UseBinary = true;
    reqFTP.Proxy = null;
    reqFTP.UsePassive = true;

    Console.WriteLine("Uploading " + file1);

    FileStream fs = File.OpenRead(@"D:\LEV\" + file1);
    byte[] buffer = new byte[fs.Length];
    fs.Read(buffer, 0, buffer.Length);
    fs.Close();
    Stream ftpstream = reqFTP.GetRequestStream();
    ftpstream.Write(buffer, 0, buffer.Length);
    ftpstream.Close();
}

static public string[] GetFileListUpload()
{
    string[] uploadFiles = Directory.GetFiles(@"D:\LEV\", "*.*", SearchOption.TopDirectoryOnly);
    return uploadFiles;
}
Dan Sewell
  • 1,278
  • 4
  • 18
  • 45
  • 1
    what is your internet speed ? does it have guaranteed or best-effort bandwidth ? – cristi _b Dec 25 '12 at 14:54
  • Hi, Good point. I am on an Azure 'small' Virtual machine, so I assume the bandwidth isn't a limiting factor here. – Dan Sewell Dec 25 '12 at 14:55
  • 1
    well you might have a good internet connection, but what about the remote ftp server where you are trying to upload files ? – cristi _b Dec 25 '12 at 14:57

1 Answers1

5

There are several factors to consider here:

  • Your internet connection is not guaranteed to be symmetrical. Most internet connection plans (in my area, at least) offer an upload bandwidth which is 1/8th of the download bandwidth.

  • The FTP server itself may be limiting the bandwidth of incoming connections.

  • The FTP server may also be throttling the maximum bandwidth per upload. In this case, you will benefit greatly from multithreading the upload, uploading many files at once.

Rotem
  • 21,452
  • 6
  • 62
  • 109
  • I have just added an edit. I have just done another test on a different VM (non azure) using the windows ftp.exe and it is 2 times quicker than my console app. – Dan Sewell Dec 25 '12 at 15:06
  • 1
    To understand if there is something specific in your code causing the slowdown, perhaps try using `WebClient` to do the uploads, and see if you are getting better results. – Rotem Dec 25 '12 at 16:25