Using wildcard in remote path using Paramiko's SFTPClient

Question

I want to copy the file from remote server to local.

import paramiko
paramiko.util.log_to_file('/tmp/paramiko.log')

# open transport
username = "user"
host="example.com"
port = 22
transport = paramiko.Transport((host, port))
transport.start_client()
private_key_file = "/home/user/.ssh/id_rsa"
agent = paramiko.Agent()
key = paramiko.RSAKey.from_private_key_file(private_key_file)
transport.auth_publickey(username, key)

# get sftp client
sftp = paramiko.SFTPClient.from_transport(transport)
source = "/home/user/user_1.csv"
target = "/home/local/local_sftp.txt"
sftp.get(x[0], x[1])

Above code is working fine but I want to use source = "/home/user/user_*.csv" but this wildcard is not evaluated. Can someone please help me fix this problem.

I found one solution for SCPClient but not able to fix the problem for SFTPClient.

Martin Prikryl · Accepted Answer · 2021-03-11T09:09:03.803

Paramiko SFTPClient does not support wildcards.

So you have to list files in a remote folder yourself and filter them to those you want to download:

import re

remote_path = "/home/user"
local_path = "/home/local"

files = sftp.listdir(remote_path)

for filename in files:
    if re.match("^user_.*\\.csv$", filename):
        print(filename)
        sftp.get(remote_path + "/" + filename, local_path + "/" + filename)

Or use the fnmatch module. See List files on SFTP server matching wildcard in Python using Paramiko.

score 3 · Answer 2 · answered Dec 11 '18 at 04:03

3

After retrieving the list of files, fnmatch is usually better than regex expression for this job:

import fnmatch
for name in sftp.listdir(remote_path):
    if fnmatch.fnmatch(name, "user_*.csv"):
        print(name)

answered Dec 11 '18 at 04:03

Provi

66
9

I tried your solution but still, it failed, Filename is like `Opt_adv048-2020.12.23.14.05.51.csv.pgp` . I tried pattern as `Opt_adv048-*.csv.pgp` – Kar Dec 24 '20 at 21:06

Hossein Torabi · Answer 3 · 2021-04-15T14:19:23.323

0

Just an Easy soloution if you the SFTPServer deployed on the *nix like systems, use execute method like this:

conn.execute("ls dir/*.csv")

edited Apr 15 '21 at 14:19

answered Apr 14 '21 at 14:02

Hossein Torabi

694
1
7
18

That assumes you have a shell access to the server and that it's a Linux server. While this question is about SFTP. – Martin Prikryl Apr 14 '21 at 16:08
@MartinPrikryl SFTP is the abbreviation of SSH File Transfer Protocol, so basically if you have SFTP you have a Linux server, maybe you mean FTPS! – Hossein Torabi Apr 14 '21 at 17:44
Absolutely not. There are SFTP servers for all systems, not only for Linux. And even on Linux, you can have an SFTP access without a shell access. – Martin Prikryl Apr 14 '21 at 17:58
Yup but in actual question its obvious he is using linux and linux is not vital! Using shell and bash is important – Hossein Torabi Apr 14 '21 at 21:21
But we do not know if OP has a shell access. I'm not saying that your answer is not useful. But the question is about [tag:sftp]. Your answer does not use SFTP, but shell. So it should say it, so it's clear that not everyone will be able to use your solution – Martin Prikryl Apr 15 '21 at 06:20

Using wildcard in remote path using Paramiko's SFTPClient

3 Answers3

Linked