0

Hi I am trying to transfer a list of files from the server to my computer using a while loop, but it keeps giving me the error that there is no file or directory. I can't understand why.

So I have this file list_numbers.txt with over 500 lines

list_numbers.txt
1234
345
2135
2132
...

And I want to re-iterate through this list to transfer files from server to my computer with the corresponding foldername to my computer but replacing assembly_graph.txt with $line_for_assembly.txt

while read line; do scp -r user@server.com:/home/Documents/$line_assembly/assembly_graph.txt /Users/Documents/$line_for_assembly.txt; done < list_numbers.txt

So basically I want it to do the below code, but instead of me having to manually type it, it would re-iterate through the list.

scp -r user@server.com:/home/Documents/1234_assembly/assembly_graph.txt /Users/Documents/1234_for_assembly.txt

scp -r user@server.com:/home/Documents/345_assembly/assembly_graph.txt /Users/Documents/345_for_assembly.txt

scp -r user@server.com:/home/Documents/2135_assembly/assembly_graph.txt /Users/Documents/2135_for_assembly.txt

scp -r user@server.com:/home/Documents/2132_assembly/assembly_graph.txt /Users/Documents/2132_for_assembly.txt

But I get this error:

scp: /home/Documents//assembly_graph.txt: No such file or directory

So it is not picking up the $line_assembly, even when I do ${line}_assembly. I don't know what I'm doing wrong.

cms72
  • 177
  • 10
  • If you notice, `$line_assembly` is not being processed. There should be some directory between those two forward slash `//` between *Documents* and *assembly_graph.txt*. – Mayura Jun 04 '19 at 02:57
  • Please add what input file looks like and the directory structure of target file system. Also an explination of exactly what youre trying to do. It looks like youre trying to take input from a file and use those lines in scp as a directory path. Does that sound right. – goose goose Jun 04 '19 at 03:02
  • Hi goose goose, I updated the question. – cms72 Jun 04 '19 at 03:29
  • When you do it manually does it work ? – goose goose Jun 04 '19 at 03:40
  • Yes. It works manually – cms72 Jun 04 '19 at 03:42
  • 1
    Try quoting the variable like user@server.com:/home/Documents/"$line"_assembly/assembly_graph.txt /Users/Documents/"$line"_for_assembly.txt; done < list_numbers.txt – goose goose Jun 04 '19 at 03:53
  • Thanks goose! The quotes worked, but I think there's something wrong with the list of files I'm giving it, its picking up the numbers in the list_numbers.txt but still saying there is not filename blah blah. So I'll look into that. But at least its reading it now. Thank you! – cms72 Jun 04 '19 at 04:13
  • The next thing is because the syntax is scp host then destination the part you have for host is /Users/Documents/$line_for_assembly.txt has to exist furthermore and most importantly on the destination be sure to use the user@ syntax and the directory /Users/Documents exists – goose goose Jun 04 '19 at 04:34
  • 1
    scp [OPTION] [user@]SRC_HOST:]file1 [user@]DEST_HOST:]file2 – goose goose Jun 04 '19 at 04:37
  • Ohhhh I see. I'll give that a go. Thanks goose goose! – cms72 Jun 04 '19 at 04:58
  • Did you see the "this has already been answered" part. Up top. Thats the problem solver – goose goose Jun 04 '19 at 05:37
  • I did but it wasn't doing it for me. Then GMc gave the idea that there could be characters in my list! Its working now. Thank you both! – cms72 Jun 04 '19 at 12:43

1 Answers1

0

because you do not have a variable named line_assembly.

Perhaps you mean to the following:

while read line
 do
  line=`echo $line | tr -d '\r\n'`
   scp -r user@server.com:/home/Documents/${line}_assembly/assembly_graph.txt /Users/Documents/${line}_for_assembly.txt
done < list_numbers.txt

this will expand as the value of line followed by the text "_assembly".

For example if "line" has the value "component" you will get

.../Documents/component_assembly/...

EDIT: OP had spurious characters in his/her input file (probably \r characters from windows). I've edited the answer accordingly to accommodate OP's specific issue.

GMc
  • 1,764
  • 1
  • 8
  • 26
  • Hi GMc, I tried this, but its giving me this error now ```_assembly/assembly_graph.txt: No such file or directory``` – cms72 Jun 04 '19 at 03:31
  • 1
    I guess I can not see any of your directories, so I can only ask does that file exist on the remote (or local) system? Also, I noticed in my answer I did not properly "escape" the destination path which will have the exact same problem, so I will update my answer accordingly. – GMc Jun 04 '19 at 05:18
  • 1
    Also, you might need to make sure your input file does not have any spurious characters. You might try adding ```line=`echo $line | tr -d '\r\n'` ``` just before the scp – GMc Jun 04 '19 at 05:26
  • OMG. It worked. It was the line=`echo $line | tr -d '\r\n'` that did the trick. I guess there were extra characters I didn't see. Its been doing my head in! Thank you so much GMc!!! – cms72 Jun 04 '19 at 12:26
  • You are most welcome and thanks for accepting my answer (most do not - even those that log a comment that said it solved their problem). BTW. For future reference, you can see the "detailed contents of a file with ```hexdump -C list_numbers.txt``` Try it on your file. Normally, Linux text files end their line with a single linefeed character which will show as 0A in hexdump. Windows text files use two characters which will show as 0D 0A in hexdump. – GMc Jun 04 '19 at 22:28
  • You saved me hours of having to manually type scp for each file, so really - thank you! Ah, thats a handy one-liner file check! I think because I saved it as a text file from an xlsx file so it must have picked up extra characters. Good to know! – cms72 Jun 06 '19 at 12:23