2

I want to download or read part of a file from the FTP server instead of downloading the whole file, this is to just see the data that exists in the FTP server is correct.

We have so many clients and each file in the FTP server might be of any size, so instead of downloading or reading the complete file, I just want to download or read a part of the file, let's say I want only 5kb of file or if by line 100 lines from a file.

I have a function in PHP like below which does half of the work, but for larger files, it fails.

function readByBytes($path)
{
    try
    {
        $handle = fopen($path, "rb");
        if ($handle) 
        {
            while (($buffer = fgets($handle, 4096)) !== false)
            {

            }
            if (!feof($handle))
            {
                echo "Error: unexpected fgets() fail\n";
            }
            fclose($handle);
        }
    }
    catch (Exception $e)
    {
        echo $e;
    }
}


$filename = "ftp://username:password@127.0.0.1/prod/clientfeed.csv";

$iterator = readByBytes($filename);

foreach ($iterator as $key => $iteration)
{
    /// if file read is 5kb or some 100 lines
    break;
}

Can somebody help me or guide me on this in PHP or Python

Below warning errors getting

PHP Warning:  fopen(ftp://...@127.0.0.1/prod/clientfeed.csv): failed 
to open stream: FTP server reports 550 Could not get file size.
 in /var/www/html/newLpplugins/ftp_read_line_line.php on line 80
PHP Warning:  filesize(): stat failed for 
ftp://...@127.0.0.1/prod/clientfeed.csv in 
/var/www/html/newLpplugins/ftp_read_line_line.php on line 81
PHP Warning:  fread() expects parameter 1 to be resource, bool given 
in /var/www/html/newLpplugins/ftp_read_line_line.php on line 81
PHP Warning:  fclose() expects parameter 1 to be resource, bool given 
in /var/www/html/newLpplugins/ftp_read_line_line.php on line 82
PHP Warning:  
file_get_contents(ftp://...@127.0.0.1/prod/clientfeed.csv): failed to 
open stream: FTP server reports 550 Could not get file size.
in /var/www/html/newLpplugins/ftp_read_line_line.php on line 84

Thanks in advance.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
Chethu
  • 555
  • 4
  • 13

2 Answers2

5

If you want to read only part of the file, then just remove your while loop and call fgets only once.

$buffer = fgets($handle, 4096);

Though if the file is binary or if you want to read a fixes amount of bytes, you better use fread.

$buffer = fread($handle, 4096);

Though your server is not compatible with PHP URL wrappers, see:
Getting "FTP server reports 550 Could not get file size." when using FTP URL in fopen
And PHP does not offer any other robust alternative for your needs.

Though it is doable in Python with ftplib:

ftp = FTP()
ftp.connect(host, user, passwd)

size = 4096

cmd = "RETR {}".format(filename)
f = BytesIO()
aborted = False

def gotdata(data):
    f.write(data)
    while (not aborted) and (f.tell() >= size):
        ftp.abort()
        aborted = True

try:
    ftp.retrbinary(cmd, gotdata)
except:
    # An exception when transfer is aborted is expected
    if not aborted:
        raise

f.seek(0)

The code is based on my answer to:
Get files names inside a zip file on FTP server without downloading whole archive

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
0

There are ways to achieve this in python.

Solution 1:

Paramkio - The SSH V2 library implemented in python Paramiko has the option to read N bytes from the file which is present in a remote location

sClient = ssh_client.open_sftp()
remoteFileLocation = sftp_client.open('<filepath>')
for line in remote_file:
    #dosomethinghere

Soltuion 2:

This is an ad-hoc solution. Before that just clarifying if you just want to get to know that the file has content in it, you can use this.

Run a remote command to get a file size using the Python subprocess module.

        from subprocess import Popen, PIPE
        p = Popen(["du", "-sh", "filepath"], stdin=PIPE)
        output = p.communicate(msg.as_string())
Kovalan R
  • 110
  • 1
  • 1
  • 6