Curl download all files in a directory

Question

I am using PHP Curl to grab files from a url. Currently the file names are hard coded and I am trying to make it so that it downloads all the log files in a specific directory. Any point in the right direction would be nice. Thank you.

Please note that I have "Read" wright to a directory, I don't have FTP access or anything else.

Server_url : http://192.168.2.45/logfiles/
Server : server1
Files in that directory : 140512 ... 150316.log and growing


<?php
$server_url = $_GET['server_url'];
$server = $_GET['server'];
//This needs to be changed to get all files
for($i = 140512; $i <= 150316; $i++) {

    $id = base64_encode($i);
    $file_name = $server_url.$i.'.log';
    curl_setopt($ch,CURLOPT_URL,$file_name);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // return data as string

    // disable peer verification
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);

    // disable host verification
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);

    // spoof a real browser client string
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
    $output = curl_exec($ch); // capture data into $output variable
    if(!dir('sites/'.$server.'_LOGS')){
        mkdir('sites/'.$server.'_LOGS');
    }
    if( $output != false){
        file_put_contents('sites/'.$server.'_LOGS'.'/u_ex' . base64_decode($id) . '.log', $output );
    }
    curl_close($ch);
}
?>

I think that is over kill. All I want to do is copy all the files over. — Sari Rahal, Mar 16 '15 at 18:18
It seems like this is a server on your network? And you are just gathering logs right? Why not just rsync the logs? — dudeman, Mar 16 '15 at 22:24

score 0 · Answer 1 · edited May 23 '17 at 11:50

0

Curl supports ftp, you can use it to get the file list and then download each file. I found an example in a previous answer using php curl here downloading all the files in a directory with cURL.

edited May 23 '17 at 11:50

Community

1
1

answered Mar 16 '15 at 18:38

takacsmark

3,933
23
26

Unfortunately for me, FTP isn't set up on the server. – Sari Rahal Mar 16 '15 at 18:47
you should install ftp, its easier than scaping urls in a curl url call to download the files and additional code will be required. if you're on linux server, issue > apt-get / or yum install vstpd then use > wget --no-verbose --no-parent --recursive --level=1 --no-directories --user=login --password=pass ftp://ftp.myftpsite.com/ to retrieve the files. – unixmiah Mar 16 '15 at 19:23
I wish I could, I do not have the ability to install anything on the server. All I have is a read permission to a single directory. – Sari Rahal Mar 17 '15 at 14:53
Let me suggest to add more information about your environment to your question, since your access rights are limited, we can only have some wild guesses now. – takacsmark Mar 17 '15 at 14:59

Curl download all files in a directory

1 Answers1