Unzip files on an SFTP server using Python

Question

I have a zip file which is present on an SFTP server. I am writing a AWS lambda function and have Paramiko module to connect to the SFTP server. I have written below code to unzip the file and code is getting executed successfully but the file is not getting unzipped. Can some one please help where I am going wrong or an alternate way to unzip the file on server?

to_sftp.chdir('<dir name>')
flo =  BytesIO()
to_sftp.getfo('<dir name/filename.zip', flo)
flo.seek(0)
root = z.ZipFile(flo,'w')
print(root)
root.extractall(to_sftp.chdir('<dir name'))

Martin Prikryl · Answer 1 · 2023-08-23T05:45:28.137

There's no way the ZipFile.extractall can write the files directly to the SFTP server. You have to extract the files to a local temporary folder. And then upload them one-by-one.

AWS Lambda seems to have a temporary folder in /tmp, just like regular *nix server. See Can I store a temp file on AWS Lambda Function?

Other option is to explicitly iterate all files in the ZIP archive using ZipFile.infolist or ZipFile.namelist, and transfer them in-memory to the SFTP server using ZipFile.open and SFTPClient.putfo:

for i in root.infolist():
    with root.open(i) as f:
        to_sftp.putfo(f, "/remote/path/" + i.filename)

(untested, just a concept)

Alternatively, if you have a shell access to the server, run unzip (or an equivalent) directly on the server, to avoid a need to download the ZIP file and upload back the extracted files.

Note that it might be more efficient to read the ZIP archive directly from the SFTP server:

with to_sftp.open('<dir name/filename.zip', bufsize=32768) as f:
    root = z.ZipFile(f, 'r')

I also assume the r mode is more appropriate for your task.
For the purpose of the bufsize=32768, see Reading file opened with Python Paramiko SFTPClient.open method is slow.

Unzip files on an SFTP server using Python

1 Answers1