Paramiko scp copy from remote machine regular expression

Question

Is there way i can copy remote files that ends with name "output" using paramiko scp.

I have below code, which copies only if i provide full path or exact file name

Below is code

 import paramiko
 import os
 from paramiko import SSHClient
 from scp import SCPClient


def createSSHClient(self, server):
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(server, self.port, self.user, self.password)
    return client

  def get_copy(self, hostname, dst):
    ssh = self.createSSHClient(hostname)
    scp = SCPClient(ssh.get_transport())
    scp.get(dst)
    scp.close()

What am trying is

     get_copy(1.1.1.1, "*output")

I am getting file not found Error

Perhaps little more detail about what you want to achieve would be great. From what I understand, paramiko can run `ls ${dst} -p | grep -v /` to get filenames, and iterate over it with `if name.endswith('output'): ...`, and so on — Chris, Oct 15 '18 at 04:37
Hi Chris, Sorry my question is confusing. I corrected in post so others will understand clearly — Dev Ops Ranga, Oct 15 '18 at 05:50
See also [Using wildcards in file names using Python's SCPClient library](https://stackoverflow.com/q/47926123/850848). — Martin Prikryl, Sep 08 '20 at 09:39

score 5 · Accepted Answer · answered Oct 15 '18 at 04:52

5

Maybe need to use ssh to get list first, then scp them one by one.

Something like follows, just FYI.

def get_copy(self, hostname, dst):
    ssh = createSSHClient(hostname)

    stdin, stdout, stderr = ssh.exec_command('ls /home/username/*output')
    result = stdout.read().split()

    scp = SCPClient(ssh.get_transport())
    for per_result in result:
        scp.get(per_result)
    scp.close()
    ssh.close()

answered Oct 15 '18 at 04:52

atline

28,355
16
77
113

While this works, you can also filter the files server-side, what is more efficient. See [Using wildcards in file names using Python's SCPClient library](https://stackoverflow.com/q/47926123/850848). Of course, only when the shell wildcards work for your filtering needs. – Martin Prikryl Sep 08 '20 at 09:40

score 1 · Answer 2 · answered Apr 25 '19 at 04:01

There are two more ways which I found useful in this context.

1) You may also do it without using SCPClient, but just Paramiko itself. Like -

def get_copy(self, hostname, dst):
    ssh = createSSHClient(hostname)
    sftp = ssh.open_sftp()
    serverfilelist = sftp.listdir(remote_path)
    for f in serverfilelist:
            if re.search("*output", f):
                sftp.get(os.path.join(remote_path, f), local_path)

    ssh.close()

2) If you want to use SCPClient to SCP files using regex(wildcards), THIS link will be helpful, I think.

Paramiko scp copy from remote machine regular expression

2 Answers2

Linked

Related