7

I want to copy files from remote machine using Python script to my local machine. I only know the extension of the file names so I want to use wildcards to express the file name.

In addition, I want to use the SCPClient Python library and not the os.system directly as suggested in the question titled using wildcards in filename in scp in python

But when I run the following code:

from paramiko import SSHClient
import paramiko
from scp import SCPClient

with SSHClient() as ssh:
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh.connect('10.10.100.5', username= 'root', password='Secret')
    with SCPClient(ssh.get_transport()) as scp:
        scp.get(remote_path='/root/*.py', local_path='.')

I get an exception

scp.SCPException: scp: /root/*.py: No such file or directory

Running from shell works just fine

scp root@10.10.100.5:/root/*.py .

Itai Agmon
  • 1,417
  • 13
  • 12

1 Answers1

15

You need to add sanitize to your get_transport():

with SCPClient(ssh.get_transport(), sanitize=lambda x: x) as scp:
        scp.get(remote_path='/root/*.py', local_path='.')

Otherwise wildcards are treated literally.

EDIT Sep 2020: There has been some confusion about this answer. The above code does not sanitise anything but it provides the mandated sanitisation function. Paramiko does not have a default sanitiser we would now be turning off. Whatever sanitisation happens or does not happen must come from the implementation, as there is no way for Paramiko to know which paths are safe in your particular environment.

If there is no sanitising function present at all, all wildcards are interpreted literally - I assume for security reasons as wildcards are dangerous with untrusted input. The design forces the implementation to consider sanitisation by adding a sanitising function of some kind. If there is none, wildcards do not work as wildcards.

To make wildcards work, there needs to be a sanitising function that returns the sanitised path. The above lambda is a dummy function that returns whatever is fed to it - which means it does not do any sanitisation at all but as there now is a sanitising function, wildcards start now working as wildcards.

If you do not trust your inputs, do not do this. In that case you need to write a proper sanitiser that will make sure the path is safe to use. You will need to write a function that returns the sanitised path.

Hannu
  • 11,685
  • 4
  • 35
  • 51
  • 2
    To clarify this answer, you are turning off sanitization, not actually adding it. This is done by setting sanitize to a function that basically does nothing other than return the value as-is. – Klinky Apr 20 '20 at 22:10
  • +1 I couldn't find any hints in the docs that this has to be disabled for wildcards to work. I am not sure, however, why using a useless function fixes the problem. @Klinky If you could provide some more insight, I would be very grateful. – DocDriven Sep 04 '20 at 17:02
  • It does if you trust your inputs. Wildcards are not accepted if there is no sanitisation at all. To make them work you need to write your own sanitiser that needs to return the sanitised path. If you do not trust your input, you need to write a proper sanitising function and use that instead. I have never had to do that as I would never write code that executes scp based on user input, so the dummy lambda has always been enough. But I should have explained it more thoroughly. – Hannu Sep 05 '20 at 18:30
  • For the sake of clarity I have edited my answer with a bit more context. I acknowledge the answer was not clear at all. It works but it failed to explain why and what are the important caveats. BTW - this does not "turn off" sanitisation as by default there is none. – Hannu Sep 07 '20 at 10:10