I am trying to copy all csv files from the remote sftp server and paste those files to the local server in a loop. Then I would like to create a listing of those files as a .txt file in order to process those files. I tried some existing solutions for that purpose, however none of them works properly. I am quite newbie in bash scripting.
Thank you in advance.
1) You can create rsa-keys to link your remote server and your local server. That way you won't have to put in a password:
https://www.tecmint.com/ssh-passwordless-login-using-ssh-keygen-in-5-easy-steps/
https://help.github.com/en/articles/error-agent-admitted-failure-to-sign
How can I define a bash alias as a sequence of multiple commands?
I used these three links to successfully set up rsa-keys. I created a page on my lab's wiki on how to do this step by step, if you're having issues I could probably upload the instructions to my github.
2) For copying, try this:
scp -r username@remote_server:/path_to_directory_containing_files_you_want/*csv local_path_you_want_to_copy_to
ls > name_of_file.txt
The -r means you are recursively copying all the files you specify, here they are all the csv files. You should already have your username and the remote server name. If you are already in the local directory you want to copy the files into, doing . is the same as putting in the whole local path. Do the ls command after you have copied all the files, it will create a new txt file with all the names of the files in that directory. Assuming there are only newly copied files there, this will be the easiest way to get a list of the files you copied. Hope this helps!
