477

I'd like to copy files from/to remote server in different directories. For example, I want to run these 4 commands at once.

scp remote:A/1.txt local:A/1.txt
scp remote:A/2.txt local:A/2.txt
scp remote:B/1.txt local:B/1.txt
scp remote:C/1.txt local:C/1.txt

What is the easiest way to do that?

user987654
  • 5,461
  • 5
  • 23
  • 26
  • 22
    When I made a script, I had to put password for each command. Can I avoid it? – user987654 Jun 02 '13 at 19:09
  • 15
    Avoid repeating password this way: scp remote:"A/1.txt A/2.txt B/1.txt C/1.txt" local:./ – JohnMudd Jul 21 '15 at 14:33
  • http://stackoverflow.com/a/23748561/874188 (nominated as duplicate of this one) has a nice additional technique. – tripleee Jul 31 '16 at 09:43
  • 3
    I would suggest that you have a look at `rsync`, maybe it can help you in this case and many upcoming cases. Then, to avoid entering passwords (let alone multiple times) you should read about `ssh` public/private keys, e.g. https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2 – mnagel Nov 11 '16 at 09:08
  • 2
    Example based on @JohnMudd 's answer: `scp root@192.168.56.120:'/etc/openvpn/ca.crt /etc/openvpn/client/client0.crt /etc/openvpn/client/client0.key /etc/openvpn/client/ta.key' ./` . – Eduardo Lucio Jul 04 '18 at 17:19

19 Answers19

608

Copy multiple files from remote to local:

$ scp your_username@remote.edu:/some/remote/directory/\{a,b,c\} ./

Copy multiple files from local to remote:

$ scp foo.txt bar.txt your_username@remotehost.edu:~
$ scp {foo,bar}.txt your_username@remotehost.edu:~
$ scp *.txt your_username@remotehost.edu:~

Copy multiple files from remote to remote:

$ scp your_username@remote1.edu:/some/remote/directory/foobar.txt \
your_username@remote2.edu:/some/remote/directory/

Source: http://www.hypexr.org/linux_scp_help.php

gaoithe
  • 4,218
  • 3
  • 30
  • 38
ios.id0
  • 6,213
  • 1
  • 12
  • 10
  • 5
    The first command omitted an ending ".", and '{a,b,c}' is equal to \{a,b,c\}. – duleshi May 05 '14 at 06:04
  • 14
    I think OP is trying to copy multiple files in multiple local folders to multiple remote folders, these examples won't do the job, as they all will scp files to `~` or `./` only. – aesede Apr 28 '15 at 17:09
  • 38
    Beware: no spaces after the commas within the curly braces. I just spent 10 minutes trying to figure out why my command was not working and it was because of spaces. – deepak Sep 28 '15 at 15:42
  • 21
    @duleshi {a,b,c} is not exactly equal to \{a,b,c\} when copying from remote to local, for the first requires password for each file while the second requires password only once. – Marcelo Ventura Nov 11 '15 at 00:24
  • 24
    @MarceloVentura, @duleshi: Also, I'd like to add that using `\{a,b,c\}` will transfer the files in a single connection/batch (since they'll be expanded on the remote host), while using `{a,b,c}` will open multiple connections, and the overhead is quite noticeable when transferring many files – omninonsense Apr 09 '16 at 10:39
  • 2
    by mistake i typed, scp `your_username@remote.edu:/some/remote/directory/\{a, b, c\} ./` (_notice space after comma in `a`, `b`, `c`_) and it failed - no space in between filenames is important thing to note!! – NoobEditor Nov 20 '17 at 10:55
  • To copy, say, `*.csv` from remote to local, `scp -p a@b:/tmp/\*.csv .` works. – Noumenon Jul 23 '18 at 15:07
  • 1
    @NoobEditor thanks for pointing this out. I make the same mistake too and did not know why until I saw your comment. – yuqli Feb 13 '19 at 19:44
  • Depending on the number of files, it may be more efficient to create a tar file including all of the files you want to copy, scp the .tar file over and untar it. scp performs an independent transfer for each file, and that can be quite slow. Deciding whether that's "easier" or not is left as an exercise for the reader. – Dan J. Mar 23 '19 at 17:22
  • There is a difference between `scp user@server:{a,b,c} ...` and `scp user@server:"a b c"` (assuming a,b and c has no spaces in their names), i.e. in the former I got password prompt as many times as I have files, though it's the same server. For the latter it happened only once. – 0andriy Sep 24 '19 at 14:57
  • Thanks @deepak I just spent waaay to long trying and failing. OP can we add that to the post as an edit for clarity? – Travasaurus Aug 06 '20 at 16:39
  • I got `No such file or directory` if it try it with `{ ... }` or `\{ ... \}` inside a bash script. If I copy this command and run it inside the terminal without a bash script, it works. – Micha93 Feb 23 '21 at 10:02
  • How about `scp` multiple files with the same extension? I have tried `"{file1,file2}".pdf` and `{"file1,file2"}.pdf` but it doesn't work: `No such file or directory`. However, your first option for one file does work. But, for more than one file, how do you do that? Please help! – Pathros Jan 28 '22 at 18:43
  • 1
    Note you can use globing, however be aware ssh will login each time to do each glob `{file*,other*,stuff*}` – Dave May 18 '22 at 19:30
  • Thanks! I added the -r flag to the first command because I wanted to copy multiple folders instead of just files. It asked for the password only once. – coder_learner Aug 17 '22 at 19:02
  • 1
    For me it only worked with `-O` option which forces original SCP protocol for file transfers instead of the SFTP protocol. – ks1322 Jan 14 '23 at 10:48
325

From local to server:

scp file1.txt file2.sh username@ip.of.server.copyto:~/pathtoupload

From server to local (up to OpenSSH v9.0):

scp -T username@ip.of.server.copyfrom:"file1.txt file2.txt" "~/yourpathtocopy"

From server to local (OpenSSH v9.0+):

scp -OT username@ip.of.server.copyfrom:"file1.txt file2.txt" "~/yourpathtocopy"

From man 1 scp:

-O      Use the legacy SCP protocol for file transfers instead of the SFTP protocol.  Forcing the use of the
        SCP protocol may be necessary for servers that do not implement SFTP, for backwards-compatibility for
        particular filename wildcard patterns and for expanding paths with a ‘~’ prefix for older SFTP
        servers.

HISTORY
        Since OpenSSH 9.0, scp has used the SFTP protocol for transfers by default.
Fonic
  • 2,625
  • 23
  • 20
Gtr_py
  • 3,511
  • 1
  • 12
  • 13
  • 20
    This should be the answer on top, since the server doesn't ask for the password for each file :) – motagirl2 Jan 30 '19 at 12:57
  • 1
    Not working on client on Ubuntu 18 (zsh). Not sure where this should work. ``` (venv) ➜ git:(master) ✗ scp root@172.29..xxx.yyy:"/usr/local/bin/kubectl /root/.kube/config" /tmp/ root@172.29..xxx.yyy's password: protocol error: filename does not match request ``` – jseguillon Apr 26 '19 at 12:24
  • 4
    @motagirl2 The top answer doesn't ask for the password for each file as long as you're escaping the curly brackets with the backslash. This answer gives me an error that the file cannot be found. – Pluto Jul 02 '19 at 22:00
  • 1
    The `server to local` does not work. How it has 87 up votes, I have no clue. – Deanie Jul 07 '19 at 23:45
  • 2
    the second one works on my machines but only without the -T option – Richard DiSalvo Sep 29 '19 at 20:28
  • 1
    `-T` doesn't seem to be available on what is shown as `version 7.7.2.1` for scp via `Get-Command scp` on Windows. The command works without the `-T` though. – dtasev Jan 06 '20 at 11:16
  • It works for mac. Without -T will throw protocol error: filename does not match request. But I can't find -T at linux man page[linux](https://linux.die.net/man/1/scp) . How do you find this option. – flyingDuck Dec 27 '21 at 09:09
  • It works like a charm, for anyone it doesnt work. Check if you did miss the quotes or post your command here. – Jess Oct 24 '22 at 21:30
97

You can copy whole directories with using -r switch so if you can isolate your files into own directory, you can copy everything at once.

scp -r ./dir-with-files user@remote-server:upload-path

scp -r user@remote-server:path-to-dir-with-files download-path

so for instance

scp -r root@192.168.1.100:/var/log ~/backup-logs

Or if there is just few of them, you can use:

scp 1.txt 2.txt 3.log user@remote-server:upload-path
Jiri Kremser
  • 12,471
  • 7
  • 45
  • 72
  • Does this work if my destination is root and I'm doing the etc directory like for example: scp -r /backups/etc root@linuxbox:/ where my source is a very sparse directory with things like "hosts" and nginx/sites-available etc ? – Tomachi Apr 20 '18 at 13:21
76

As Jiri mentioned, you can use scp -r user@host:/some/remote/path /some/local/path to copy files recursively. This assumes that there's a single directory containing all of the files you want to transfer (and nothing else).

However, SFTP provides an alternative if you want to transfer files from multiple different directories, and the destinations are not identical:

sftp user@host << EOF
  get /some/remote/path1/file1 /some/local/path1/file1
  get /some/remote/path2/file2 /some/local/path2/file2
  get /some/remote/path3/file3 /some/local/path3/file3
EOF

This uses the "here doc" syntax to define a sequence of SFTP input commands. As an alternative, you could put the SFTP commands into a text file and execute sftp user@host -b batchFile.txt

alev
  • 1,334
  • 9
  • 7
  • 8
    This the only answer that actually answers the question. Thanks. I've been Googling and analyzing the `scp` manual for way too long trying to find this feature, and I guess scp doesn't have it. – sudo Sep 15 '17 at 22:14
  • 3
    This seems to be the best, most flexible answer. Note: I got tripped up until I realized that I needed to use a full path like `/home/vagrant/Code/` instead of `~/Code/`. – Ryan Aug 15 '18 at 21:57
  • This is the best answer, just don't forget to add `- r` for directories – Chris_Rands Aug 27 '18 at 12:44
  • This only works if you have public key authentication setup. – Sahil Chimnani Mar 13 '22 at 05:58
42

The answers with {file1,file2,file3} works only with bash (on remote or locally)

The real way is :

scp user@remote:'/path1/file1 /path2/file2 /path3/file3' /localPath
Vouze
  • 1,678
  • 17
  • 10
26

After playing with scp for a while I have found the most robust solution:

(Beware of the single and double quotation marks)

Local to remote:

scp -r "FILE1" "FILE2" HOST:'"DIR"'

Remote to local:

scp -r HOST:'"FILE1" "FILE2"' "DIR"

Notice that whatever after "HOST:" will be sent to the remote and parsed there. So we must make sure they are not processed by the local shell. That is why single quotation marks come in. The double quotation marks are used to handle spaces in the file names.

If files are all in the same directory, we can use * to match them all, such as

scp -r "DIR_IN"/*.txt HOST:'"DIR"'
scp -r HOST:'"DIR_IN"/*.txt' "DIR"

Compared to using the "{}" syntax which is supported only by some shells, this one is universal

Eox
  • 341
  • 3
  • 8
  • Thanks for helping with spaces in the file names. I was using {} approach so I also had to escape them with \ after quoting. – Noumenon Feb 22 '19 at 23:16
  • 3
    In case you want to use the tilde character (~) for your home directory in paths. Note how the ~ and / must be outside the quotes. ``` scp -r robwaa@192.168.1.101:'~/"bin/set-vpn-kill-switch" ~/"bin/flush-iptables"' ~/"" ``` – Rob Waa Apr 11 '19 at 16:09
  • The `{}` syntax makes it easier to copy multiple files from the same directory, is there any way to do this with `scp -r` so that `"FILE1" "FILE2"` can be relative paths and not absolute paths to the files? – dtasev Jan 06 '20 at 11:07
16

The simplest way is 

local$ scp remote:{A/1,A/2,B/3,C/4}.txt ./

So {.. } list can include directories (A,B and C here are directories; "1.txt" and "2.txt" are file names in those directories).

Although it would copy all these four files into one local directory - not sure if that's what you wanted.

In the above case you will end up remote files A/1.txt, A/2.txt, B/3.txt and C/4.txt copied over to a single local directory, with file names ./1.txt, ./2.txt, ./3.txt and ./4.txt

Tagar
  • 13,911
  • 6
  • 95
  • 110
  • 1
    Wouldn't this overwrite the content of A/1.txt with the content of B/1.txt on the local machine? – Yves Dorfsman Jun 07 '16 at 04:03
  • good point.. adjusted example (so now it wouldn't overwrite) and added description. Thanks. – Tagar Jun 07 '16 at 15:46
  • Brace expansions like this are a Bash feature, and not portable to e.g. POSIX `sh`. – tripleee Jul 31 '16 at 09:41
  • 1
    @tripleee, thanks. it might not be posix, but it works not only in bash. I use ksh all the time and it works there too. – Tagar Jul 31 '16 at 18:36
  • 1
    You can also mix dirs and files by doing: `local$ scp -r remote:{A/1.txt,A/2.txt,B/3.txt,C/4.txt,D,F} ./` – donhector Oct 19 '16 at 06:40
16

Problem: Copying multiple directories from remote server to local machine using a single SCP command and retaining each directory as it is in the remote server.

Solution: SCP can do this easily. This solves the annoying problem of entering password multiple times when using SCP with multiple folders. Consequently, this also saves a lot of time!

e.g.

# copies folders t1, t2, t3 from `test` to your local working directory
# note that there shouldn't be any space in between the folder names;
# we also escape the braces.
# please note the dot at the end of the SCP command

~$ cd ~/working/directory
~$ scp -r username@contact.server.de:/work/datasets/images/test/\{t1,t2,t3\}  .

PS: Motivated by this great answer: scp or sftp copy multiple files with single command

Based on the comments, this also works fine in Git Bash on Windows

kmario23
  • 57,311
  • 13
  • 161
  • 150
  • 7
    P.S. do not add spaces between items. i.e. \{t1, t2, t3\} ------> I wasted ten minutes here...lol – kenberkeley Dec 20 '18 at 04:58
  • 1
    This is the best answer because when you escape the braces it will be interpreted as a single command by and it won't ask for the password for each transfer – Amaynut Jul 19 '19 at 21:35
  • @Amaynut yes, exactly! That certainly saves lot of time and effort :) – kmario23 Oct 23 '19 at 13:57
12

You can do this way:

scp hostname@serverNameOrServerIp:/path/to/files/\\{file1,file2,file3\\}.fileExtension ./

This will download all the listed filenames to whatever local directory you're on.

Make sure not to put spaces between each filename only use a comma ,.

Simson
  • 3,373
  • 2
  • 24
  • 38
J_rad
  • 181
  • 1
  • 3
  • 8
  • 4
    This is good when the extension is the same. It could be written as `/path/to/files/\\{file1.ext,file2.ext,file3.ext\\}` if the files have different extensions. Note to PowerShell people (now that MS has joined the party) you need to use `\`` to escape instead: `/path/to/files/\`{file1.ext,file2.ext,file3.ext\`}` – dtasev Jan 06 '20 at 11:00
  • How about `scp` multiple files with the same extension? I have tried `"{file1,file2}".pdf` and `{"file1,file2"}.pdf` but it doesn't work: `No such file or directory`. However, your first option for one file does work. But, for more than one file, how do you do that? Please help! – Pathros Jan 28 '22 at 18:46
11

Copy multiple directories:

scp -r dir1 dir2 dir3 admin@127.0.0.1:~/
yuliskov
  • 1,379
  • 15
  • 16
5

Is more simple without using scp:

tar cf - file1 ... file_n | ssh user@server 'tar xf -'

This also let you do some things like compress the stream (-C) or (since OpenSSH v7.3) -J to jump any times through one (or more) proxy servers.

Avoid using passwords by coping your public key to ~/.ssh/authorized_keys (on server) with ssh-copy-id (on client).

Posted also here (with more details) and here.

Pablo Bianchi
  • 1,824
  • 1
  • 26
  • 30
4
scp remote:"[A-C]/[12].txt" local:
unxnut
  • 8,509
  • 3
  • 27
  • 41
  • thanks, but it is just an example, and the actual file or directory name is more complex. But I have a list of them. – user987654 Jun 02 '13 at 19:07
  • 1
    Can you describe those by a regular expression? – unxnut Jun 03 '13 at 01:43
  • Is it possible to preserve the same directory structure in target directory by using regular expression? – user987654 Jun 03 '13 at 21:36
  • 1
    Yes, you should be able to do that with `-r` option to `scp`. And you can preserve other attributes (like time stamp) with `-p` option. – unxnut Jun 04 '13 at 02:49
2

NOTE: I apologize in advance for answering only a portion of the above question. However, I found these commands to be useful for my current unix needs.

Uploading specific files from a local machine to a remote machine:

~/Desktop/dump_files$ scp file1.txt file2.txt lab1.cpp etc.ext your-user-id@remotemachine.edu:Folder1/DestinationFolderForFiles/

Uploading an entire directory from a local machine to a remote machine:

~$ scp -r Desktop/dump_files your-user-id@remotemachine.edu:Folder1/DestinationFolderForFiles/

Downloading an entire directory from a remote machine to a local machine:

~/Desktop$ scp -r your-user-id@remote.host.edu:Public/web/ Desktop/

fat43r 80ard
  • 37
  • 3
2

In my case, I am restricted to only using the sftp command.
So, I had to use a batchfile with sftp. I created a script such as the following. This assumes you are working in the /tmp directory, and you want to put the files in the destdir_on_remote_system on the remote system. This also only works with a noninteractive login. You need to set up public/private keys so you can login without entering a password. Change as needed.

#!/bin/bash

cd /tmp
# start script with list of files to transfer
ls -1 fileset1* > batchfile1
ls -1 fileset2* >> batchfile1

sed -i -e 's/^/put /' batchfile1
echo "cd destdir_on_remote_system" > batchfile
cat batchfile1 >> batchfile
rm batchfile1

sftp -b batchfile user@host
dminear
  • 111
  • 6
2

In the specific case where all the files have the same extension but with different suffix (say number of log file) you use the following:

scp user_name@ip.of.remote.machine:/some/log/folder/some_log_file.* ./

This will copy all files named some_log_file from the given folder within the remote, i.e.- some_log_file.1 , some_log_file.2, some_log_file.3 ....

Guy Avraham
  • 3,482
  • 3
  • 38
  • 50
2

In my case there were too many files with non related names.

I ended up using,

$  for i in $(ssh remote 'ls ~/dir'); do scp remote:~/dir/$i ./$i; done
1.txt                                         100%  322KB   1.2MB/s   00:00
2.txt                                         100%   33KB 460.7KB/s   00:00
3.txt                                         100%   61KB 572.1KB/s   00:00

$
user9869932
  • 6,571
  • 3
  • 55
  • 49
1

scp uses ssh for data transfer with the same authentication and provides the same security as ssh.

A best practise here is to implement "SSH KEYS AND PUBLIC KEY AUTHENTICATION". With this, you can write your scripts without worring about authentication. Simple as that.

See WHAT IS SSH-KEYGEN

Augusto
  • 2,125
  • 18
  • 27
0
serverHomeDir='/home/somepath/ftp/'
backupDirAbsolutePath=${serverHomeDir}'_sqldump_'
backupDbName1='2021-08-27-03-56-somesite-latin2.sql'
backupDbName2='2021-08-27-03-56-somesite-latin1.sql'
backupDbName3='2021-08-27-03-56-somesite-utf8.sql'
backupDbName4='2021-08-27-03-56-somesite-utf8mb4.sql'

scp -i ~/.ssh/id_rsa.pub user@server.domain.com:${backupDirAbsolutePath}/"{$backupDbName1,$backupDbName2,$backupDbName3,$backupDbName4}" .

. - at the end will download the files to current dir

-i ~/.ssh/id_rsa.pub - assuming that you established ssh to your server with .pub key

DevWL
  • 17,345
  • 6
  • 90
  • 86
0
scp -r root@ip-address:/root/dir/ C:\Users\your-name\Downloads\

the -r will let you download all the files inside the dir directory of your remote server

abderrezague mohamed
  • 201
  • 2
  • 7