To start with, my goal is to use FTP to retrieve a file and put it on multiple servers. This is intended as a backup procedure where we're taking a file and putting it onto two different backup servers then removing it from the FTP server.
In case this is relevant, the program runs as a service on a server.
I have tried two different methods and gotten 3 different errors so I will detail those. I am open to trying different libraries, whole different methods, and that's why I'm focusing on the overall goal instead of the specific code and errors.
So the first method worked successfully when I tested it locally on my dev laptop; I was able to do everything I wanted. It's worth noting that I debugged as the same domain account that the service runs on in the environments. When I deployed to test I received the error "The underlying connection was closed: The server committed a protocol violation."
I have (of course) trimmed this code down to what I believe are the relevant parts.
This gets called for each path that's being delivered to. They read similar to:
first case: \\fqdn\directory
second case: \\192.168.123.123\directory$
private bool CopyFTPToPath(string file, string path)
{
try
{
string filePath = "ftp://" + Settings["Host"].Value + "/" + Settings["Path"].Value + "/" + file;
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(filePath);
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential("user", "pass");
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(responseStream))
{
using (StreamWriter writer = new StreamWriter(path + "/" + file, false))
{
writer.Write(reader.ReadToEnd());
}
}
}
}
return File.Exists(path + "/" + file);
catch(Exception ex)
{
Utility.LogError(ErrorType.Alert, ex);
}
}
Again this is successful when I run it locally, it fails when run on the server with the same credentials, with the above protocol violation error. The error occurs when it attempts to create the StreamWriter.
So I tried using a library to handle the connection and grabbed the WinSCP library because I've seen it recommended on here frequently. The code I wrote for that is this:
private bool CopyFTPToPath(string file, string path)
{
try
{
string filePath = "ftp://" + Settings["Host"].Value + "/" + Settings["Path"].Value + "/" + file;
SessionOptions sessionOptions = new SessionOptions() {
Protocol = Protocol.Ftp,
HostName = Settings.Settings["Host"].Value,
UserName = "user",
Password = "pass"
};
using(Session session = new Session())
{
session.Open(sessionOptions);
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
TransferEventArgs result = session.GetFileToDirectory(filePath, path,false,transferOptions);
}
return File.Exists(path + "/" + file);
catch(Exception ex)
{
Utility.LogError(ErrorType.Alert, ex);
}
}
Now this block of code fails at the session.GetFileToDirectory call, throwing a System.IO.DirectoryNotFoundException on the path. In this case I'm wondering if WinSCP is unable to handle a network path as the local directory path.
Here is a sanitized stacktrace from the error, for what that's worth. If the local directory parameter is actually local it seems to work, so I think that's what's going on there.
{System.IO.DirectoryNotFoundException: \\192.***.***.***\Path\Path
at WinSCP.Session.DoGetFilesToDirectory(String remoteDirectory, String localDirectory, String filemask, Boolean remove, TransferOptions options, String additionalParams)
at WinSCP.Session.GetEntryToDirectory(String remoteFilePath, String localDirectory, Boolean remove, TransferOptions options, String additionalParams)
at WinSCP.Session.GetFileToDirectory(String remoteFilePath, String localDirectory, Boolean remove, TransferOptions options)
at Program.RoutineTasks.Backup.CopyFTPToPath(String file, String path) in C:\Projects\Program\RoutineTasks\Backup.cs:line 114}
FTP is my only option to access this file. The file is 130gb, I don't have disk space on the server that runs this to copy it local to hand out. Both of those are outside of my control.
EDIT: I have found a solution that will definitely work if I can figure out how to manage the streams better, because the files are huge so I need to break them up to prevent running out of memory.
That code is this, in place of the using Session block above
using(Session session = new Session())
{
session.Open(sessionOptions);
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
using (StreamReader reader = new StreamReader(session.GetFile(filePath, transferOptions)))
{
using (StreamWriter writer = new StreamWriter(path + "/" + file, false))
{
while (!reader.EndOfStream)
{
writer.Write(reader.Read());
}
}
}
}
