Use Perl to FTP a file behind a HTTP proxy

Question

I have written a simple Perl script to list, get the mtime and then get the files (based on mtime). This has been done using the Perl Net::FTP module that is available in RHEL5/6.

My problem is that I now have to adapt this script to transfer files that are behind a HTTP squid proxy (not a FTP proxy).

I can ftp the files if I use Linux shell commands like wget or lftp after I set up the relevant proxy config files for those programs.

Is there a way to do this in Perl without calling system(shell) commands. I have tried by setting the Net::FTP firewall option (see below) however I believe this is only for FTP proxies which we do not have or use.

My code so far is similar to this (I have left out error handling for clarity):

my $host = '111.111.111.111';
my $ftp_port = 21;
my $passive = 0; #Have also tried with passive mode on.
my $firewall = 'proxy.xx:3128';
my $debug = 1;
my $username = 'foo';
my $password = 'bah';

my $ftp = Net::FTP->new($host, Port=> $ftp_port, Passive => $passive, Firewall => $firewall, Debug => $debug);
$ftp->login($username, $password);
my @file_list = $ftp->ls;
foreach my $file (@file_list) {
    my $mod_time = $ftp->mdtm($file);
    if($mod_time + $wait_time <= time() ) {
        $ftp->get($file);
    }
}
$ftp->quit;

Edit: The errors I am receiving are as follows:

Net::FTP: new(111.111.111.111, Port => 21, Passive => 0, Firewall => proxy.xx:3128, Debug => 1)
Net::FTP>>> Net::FTP(2.77)
Net::FTP>>>   Exporter(5.63)
Net::FTP>>>   Net::Cmd(2.29)
Net::FTP>>>   IO::Socket::INET(1.31)
Net::FTP>>>     IO::Socket(1.31)
Net::FTP>>>       IO::Handle(1.28)
Net::FTP=GLOB(0x2640e60): Timeout at /etc/transfer/transfer_get.pl line 106

I get the same error if I turn passive mode on. If I do not supply the Firewall option I also get a similar timeout error:

Net::FTP: new(111.111.111.111, Port => 21, Passive => 1, Debug => 1)
main crashed: Can't ftp to 111.111.111.111: Net::FTP: connect: Connection timed out

I'm pretty certain after reading the manual on Net::FTP that it doesn't support transfers through a http proxy on its own. Just wondering what the best method to achieve both listing the mtime of the remote files and then getting them in Perl.

Thanks

Paul

What happens if you try it behind the proxy? Is there an error message? — simbabque, Jul 18 '12 at 10:18
Rather than using Net::FTP, you should probably use [LWP](https://metacpan.org/release/libwww-perl). You can use it to GET a URL that starts 'ftp://' and you can also tell it to use an HTTP proxy. — Grant McLean, Jul 18 '12 at 22:20
If you use an ftp: protocol URI, you're still going to need to define an ftp proxy. — Len Jaffe, Jul 19 '12 at 17:50
Make sure you can get through to port 20. You might have firewall issues. — dadinck, Jul 19 '12 at 18:11
Have you tried setting the http_proxy/ftp_proxy environment variable? — Stefan Schroeder, May 06 '14 at 03:42

Use Perl to FTP a file behind a HTTP proxy

0 Answers0