222

I need to transfer a log file to a remote host using sftp from a Linux host. I have been provided credentials for the same from my operations group. However, since I don't have control over other host, I cannot generate and share RSA keys with the other host.

So is there a way to run the sftp command (with the username/password provided) from inside the Bash script through a cron job?

I found a similar Stack Overflow question, Specify password to sftp in a Bash script, but there was no satisfactory answer to my problem.

Community
  • 1
  • 1
anubhava
  • 761,203
  • 64
  • 569
  • 643

12 Answers12

225

You have a few options other than using public key authentication:

  1. Use keychain
  2. Use sshpass (less secured but probably that meets your requirement)
  3. Use expect (least secured and more coding needed)

If you decide to give sshpass a chance here is a working script snippet to do so:

export SSHPASS=your-password-here
sshpass -e sftp -oBatchMode=no -b - sftp-user@remote-host << !
   cd incoming
   put your-log-file.log
   bye
!
Community
  • 1
  • 1
anubhava
  • 761,203
  • 64
  • 569
  • 643
  • 3
    For using it on Mac: http://stackoverflow.com/questions/9102557/sftp-command-in-bash-script-with-mac-os-x/9102723#9102723 – anubhava Feb 27 '12 at 14:18
  • Hi, I have similar problem, and I tried to use `expect` as other alternatives are not present on the host. With `expect` I am getting this error.. `Permission denied (publickey,keyboard-interactive). `. Please can you let me know, how to connect to a sftp host with passwd from a bash script. – mtk Sep 20 '12 at 07:39
  • Please see, have asked the question here.. http://stackoverflow.com/q/12508206/1135954 – mtk Sep 20 '12 at 07:53
  • 32
    No need to export, I think. `SSHPASS=password sshpass -e ...` should do. – Jens Mar 28 '13 at 13:10
  • @Jens: Agreed that export is not really required here. – anubhava Mar 28 '13 at 13:11
  • 15
    I was able to do a one-liner to dowload a log file: `sshpass -p "my_password" sftp -oPort=9999 user@host:dir/file.log` – Cloud Artisans Jun 18 '13 at 20:24
  • 8
    @gorus: One-liner is very cool however I didn't want to use password on command line because of enhanced snooping risks. – anubhava Jun 18 '13 at 20:28
  • @Dru: Yes `export SSHPASS=your-password-here` goes into env of that user but how will `ps -ef` reveal env of a different user? – anubhava Aug 22 '13 at 08:15
  • 10
    `-oBatchMode=no` was the crucial missing piece for me. – richardkmiller Jun 06 '14 at 04:58
  • What did you use and what didn't work. Feel free to create a new question since this is very old thread. – anubhava Jul 02 '14 at 19:46
  • Hi Anubhava - Here is my question. http://stackoverflow.com/questions/24540156/automating-sftp-using-ibm-aixunix-shell-script – Teja Jul 03 '14 at 13:52
  • I am working on IBM AIX and trying to get a file from remote server using sftp command... – Teja Jul 03 '14 at 13:52
  • Hi Anubhava - Can you help me out regarding this script... I tried using except but it doesn't work. Here is my code I have written at below link. http://stackoverflow.com/questions/24540156/automating-sftp-using-ibm-aixunix-shell-script – Teja Jul 11 '14 at 14:31
  • Hi Anubhava - Any luck? – Teja Jul 11 '14 at 16:00
  • Hi Anubhava - I tried all the three options but none of them worked for me....Can you please help me out... thank you... – Teja Jul 15 '14 at 15:12
  • What error did you get while trying `sshpass` command? I use it every day on my system. – anubhava Jul 15 '14 at 15:13
  • I don't have root user on that machine to install new software. I am looking for a way to do it without installing new software – Dejell Nov 20 '14 at 16:39
  • I have installed it without root access but discussing that will be diverging too much from this question. – anubhava Nov 20 '14 at 16:40
  • Hi, sshpass is a great idea, thanks, works fine! But now, I get a mail with all SFTP-commands in the here script area every time this is run via crontab. So does anyone know how to suppress this output to the console? – spackmat Sep 08 '15 at 13:42
  • 2
    You can add `> ~/file.out 2>&1` to redirect the output and stderr to a file. – anubhava Sep 08 '15 at 13:46
  • 1
    @anubhava very thanks for this answer. I used it with my bash script file that is start via crontab -e. I change the exclamation point "!" to "EOF" because crontab doens't recognied this character in script – e2a Sep 30 '16 at 16:21
  • May I ask what exactly does the ! do? – Paxi1337 Mar 31 '21 at 08:43
  • 1
    `!` can be any marker string. You can use `EOF` or `EOD` as well. – anubhava Mar 31 '21 at 09:21
  • Hey, how can I use that in order to transfer a folder? ```put -r source target```? – Uwe.Schneider Nov 04 '22 at 09:55
  • You can zip the folder and transfer zip file for faster transfer or else use `scp` – anubhava Nov 04 '22 at 10:17
  • `keychain` won't allow you to go with a password (the specific topic of this question), but facilitates public key authentication (with a passphrase on SSH a private key), right? – rookie099 Jun 01 '23 at 03:37
132

Another way would be to use lftp:

lftp sftp://user:password@host  -e "put local-file.name; bye"

The disadvantage of this method is that other users on the computer can read the password from tools like ps and that the password can become part of your shell history.

A more secure alternative which is available since LFTP 4.5.0 is setting the LFTP_PASSWORD environment variable and executing lftp with --env-password. Here's a full example:

export LFTP_PASSWORD="just_an_example"
lftp --env-password sftp://user@host  -e "put local-file.name; bye"

# Destroy password after use
export LFTP_PASSWORD=""

LFTP also includes a cool mirroring feature (can include delete after confirmed transfer --Remove-source-files):

lftp -e 'mirror -R /local/log/path/ /remote/path/' --env-password -u user sftp.foo.com
Yvan
  • 2,539
  • 26
  • 28
Karassik
  • 1,379
  • 1
  • 8
  • 7
  • 9
    +1 for suggesting an alternative but can we avoid supplying password on command line? – anubhava Jul 24 '13 at 06:57
  • 2
    Downvoted for showing the password to anyone on the same computer – Aaron Digulla Sep 30 '15 at 13:37
  • 16
    You can create the script file for lftp, this way you will not have to provide the password in the command line. Create a file `put-script`: `open sftp://user:password@host; put local-file.name; exit` Than run `lftp -f put-script` This way you do not have to have the username and password in a command line and can set up restrictive permissions to your script file. – obaranovsky Nov 06 '15 at 01:03
  • @obaranovsky That is a good idea. I wonder if I were to add this to my environment variable as alias or a function, would it be secure and possible to hide it from other users? –  Feb 10 '17 at 05:37
  • 4
    Far easier to do `lftp` than mess around with `sftp` and `sshpass`. Good answer. – MeanEYE May 06 '18 at 14:32
  • 4
    @AaronDigulla Upvoted because it is exactly what the OP asked for - "a way to run the sftp command (**with** the username / **password provided**) from inside the Bash script". – Honza Zidek May 27 '20 at 14:29
  • 1
    Just a note, it only ran for us on Solaris if we used: *export LFTP_PASSWORD=${our_password}*, would not work without the export clause. – access_granted Aug 11 '21 at 20:45
  • @AaronDigulla I've changed the script to reflect your thoughts. Thanks for pointing this out! – Yvan Dec 28 '21 at 14:14
  • I've forgotten how often I used to use lftp [every day] and this answer completely solved an issue a client had. Also respects here docs for commands. $ lftp sftp://user@example.com << EOF ls cd foo get bar EOF – Dan Garthwaite Jun 16 '22 at 12:49
  • How do I transfer a folder with this? In the line ```"put local-file.name; bye"``` the destination path seems to be missing. – Uwe.Schneider Nov 04 '22 at 10:24
67

EXPECT is a great program to use.

On Ubuntu install it with:

sudo apt-get install expect

On a CentOS Machine install it with:

yum install expect

Lets say you want to make a connection to a sftp server and then upload a local file from your local machine to the remote sftp server

#!/usr/bin/expect

spawn sftp username@hostname.com
expect "password:"
send "yourpasswordhere\n"
expect "sftp>"
send "cd logdirectory\n"
expect "sftp>"
send "put /var/log/file.log\n"
expect "sftp>"
send "exit\n"
interact

This opens a sftp connection with your password to the server.

Then it goes to the directory where you want to upload your file, in this case "logdirectory"

This uploads a log file from the local directory found at /var/log/ with the files name being file.log to the "logdirectory" on the remote server

alexherm
  • 1,362
  • 2
  • 18
  • 31
rezizter
  • 4,908
  • 4
  • 24
  • 32
29

You can use lftp interactively in a shell script so the password not saved in .bash_history or similar by doing the following:

vi test_script.sh

Add the following to your file:

#!/bin/sh
HOST=<yourhostname>
USER=<someusername>
PASSWD=<yourpasswd>

cd <base directory for your put file>

lftp<<END_SCRIPT
open sftp://$HOST
user $USER $PASSWD
put local-file.name
bye
END_SCRIPT

And write/quit the vi editor after you edit the host, user, pass, and directory for your put file typing :wq .Then make your script executable chmod +x test_script.sh and execute it ./test_script.sh.

Mike S.
  • 4,806
  • 1
  • 33
  • 35
18

I was recently asked to switch over from ftp to sftp, in order to secure the file transmission between servers. We are using Tectia SSH package, which has an option --password to pass the password on the command line.

example : sftp --password="password" "userid"@"servername"

Batch example : 

(
  echo "
  ascii
  cd pub
  lcd dir_name
  put filename
  close
  quit
    "
) | sftp --password="password" "userid"@"servername"

I thought I should share this information, since I was looking at various websites, before running the help command (sftp -h), and was i surprised to see the password option.

nobody
  • 19,814
  • 17
  • 56
  • 77
Mahony
  • 205
  • 2
  • 2
  • 10
    +1 for you good suggestion. However this will require you to pass the password on command line and anybody on the system doing `ps` command would be able to see your password. – anubhava Apr 05 '13 at 11:54
  • 5
    I tried that, I got `unknown option -- - usage: sftp [-1246aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher] [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-R num_requests] [-S program] [-s subsystem | sftp_server] host sftp [user@]host[:file ...] sftp [user@]host[:dir[/]] sftp -b batchfile [user@]host` – code-8 Oct 24 '17 at 13:39
  • 2
    @ihue probably because you need **Techia SSH**, not OpenSSH – qris Mar 22 '18 at 11:59
  • Yeah this doesn't work with openssh, not a good solution. – Dave Apr 17 '23 at 19:49
18

You can override by enabling Password less authentication. But you should install keys (pub, priv) before going for that.

Execute the following commands at local server.

Local $> ssh-keygen -t rsa

Press ENTER for all options prompted. No values need to be typed.

Local $> cd .ssh
Local $> scp .ssh/id_rsa.pub user@targetmachine:
Prompts for pwd$>  ENTERPASSWORD

Connect to remote server using the following command

Local $> ssh user@targetmachine
Prompts for pwd$> ENTERPASSWORD

Execute the following commands at remote server

Remote $> mkdir .ssh
Remote $> chmod 700 .ssh
Remote $> cat id_rsa.pub >> .ssh/authorized_keys
Remote $> chmod 600 .ssh/authorized_keys
Remote $> exit

Execute the following command at local server to test password-less authentication. It should be connected without password.

$> ssh user@targetmachine
Eric Leschinski
  • 146,994
  • 96
  • 417
  • 335
Srini V
  • 11,045
  • 14
  • 66
  • 89
  • 2
    +1 for your answer but question was about doing it without public/private keys. – anubhava Feb 20 '14 at 06:37
  • oops.. Just saw the restrictions on key sharing :) – Srini V Feb 20 '14 at 06:41
  • Out of curiosity will this command `lftp -p ${port} -u ${login_id},${password} ${ip_number}` when invoked from a shell script print them some where? How did you solve apart from sshpass? – Srini V Feb 20 '14 at 06:43
  • I don't have `lftp` and rely on `sshpass` from Mac and Linux both. – anubhava Feb 20 '14 at 06:47
  • Pure gold. Thanks! :) – Nikolay Tsenkov Mar 13 '15 at 16:32
  • Tip: If you are setting this up for additional remote servers, copy the same id_rsa.pub to the remote servers. The public key id_rsa.pub is generated once on the local server and shared with all remote hosts with which password less authentication needs to be set up. – rao Aug 23 '17 at 18:35
13

The easiest way I found to accomplish this, without installing any third-party library like Expect, SSHPASS...etc, is by using a combination of CURL, and SFTP. Those two are almost in every Linux machine.

This is the command you should execute, after changing the values.

curl  -k "sftp://SERVER_IP:SERVER_PORT/FULL_PATH_OF_THE_FILE" --user "SERVER_USER:SERVER_PASSOWRD" -o "THE_NAME_OF_THE_FILE_AFTER_DOWNLOADING_IT"

Example:

curl  -k "sftp://10.10.10.10:77/home/admin/test.txt" --user "admin:123456" -o "test.txt"

Explanation:

We are connecting to the server 10.10.10.10:77 using the username admin and password 123456, to move the file /home/admin/test.txt from that server to the server you are using currently to execute the above command.

ASammour
  • 865
  • 9
  • 12
8

Combine sshpass with a locked-down credentials file and, in practice, it's as secure as anything - if you've got root on the box to read the credentials file, all bets are off anyway.

Rich Harding
  • 645
  • 6
  • 14
7

Bash program to wait for sftp to ask for a password then send it along:

#!/bin/bash
expect -c "
spawn sftp username@your_host
expect \"Password\"
send \"your_password_here\r\"
interact "

You may need to install expect, change the wording of 'Password' to lowercase 'p' to match what your prompt receives. The problems here is that it exposes your password in plain text in the file as well as in the command history. Which nearly defeats the purpose of having a password in the first place.

Eric Leschinski
  • 146,994
  • 96
  • 417
  • 335
5

You can use sshpass for it. Below are the steps

  1. Install sshpass For Ubuntu - sudo apt-get install sshpass
  2. Add the Remote IP to your known-host file if it is first time For Ubuntu -> ssh user@IP -> enter 'yes'
  3. give a combined command of scp and sshpass for it. Below is a sample code for war coping to remote tomcat sshpass -p '#Password_For_remote_machine' scp /home/ubuntu/latest_build/abc.war #user@#RemoteIP:/var/lib/tomcat7/webapps
ravi ranjan
  • 5,920
  • 2
  • 19
  • 18
2

A few people have mentioned sshpass but not many clear coding examples...

This is how we are doing it with bash scripts for rsync backups:

sshpass -p "${RSYNC_PASSWORD}" sftp "${RSYNC_USER}"@"${RSYNC_REMOTE_HOST}"

Keep in mind you will have to sudo apt install sshpass before this works properly.

Jesse Nickles
  • 1,435
  • 1
  • 17
  • 25
0

You can use a Python script with scp and os library to make a system call.

  1. ssh-keygen -t rsa -b 2048 (local machine)
  2. ssh-copy-id user@remote_server_address
  3. create a Python script like:
    import os
    cmd = 'scp user@remote_server_address:remote_file_path local_file_path'
    os.system(cmd)
  1. create a rule in crontab to automate your script
  2. done
Rodrigo
  • 103
  • 2
  • 6