Store output of grep command executed remotely using ssh to a variable?

Question

I am trying to run a series of commands on a remote server from a jenkins machine. Those commands do below stuff

1. Create a directory on remote server
2. Copy some files from remote server into that directory

I have created below script,

    servername=<remote server ip>

    sshuserconnect() {
        echo `/usr/bin/sshpass -p passwd ssh -o ConnectTimeout=5 -o UserKnownHostsFile=knownhosts -o StrictHostKeyChecking=no $*`
    }

sshuserconnect user@$servername << ENDOFSSH

    sqlplus -s /nolog << EOF > /home/user/output.txt
    connect user/pwd@$servername:<port>/<DB schema>
    select column1||','||value from table where column1 like 'param1';
    EOF

    version=`grep version /home/user/output.txt | cut -f 2 -d ","`
    mkdir /home/user/${version}

ENDOFSSH

Script fails with below error,

grep: output.txt: No such file or directory
Pseudo-terminal will not be allocated because stdin is not a terminal.

After some debugging I found out that script fails only for grep command. Assigning grep output to version variable fails.

I have tried lot of options and I am not if this is the correct way. Can anyone help me to resolve this?

Answer 1

In your script expansion of `grep version ...` is performed by the local shell rather than by the remote shell. The same applies to the expansion of /home/user/${version}. Subtleties of using expansions in here-documents are covered in my answer to the canonical question addressing problems of this kind.

Your script belongs to the mixed case "3. Some expansions must be performed in the child shell, some - in the parent." discussed therein, and must be fixed as follows:

sshuserconnect user@$servername <<ENDOFSSH

sqlplus -s /nolog <<EOF > /home/user/output.txt
connect user/pwd@$servername:<port>/<DB schema>
select column1||','||value from table where column1 like 'param1';
EOF

version=\$(grep version /home/user/output.txt | cut -f 2 -d ",")
mkdir /home/user/\${version}

ENDOFSSH

Note that I intentionally removed the indentation of the script section inside the <<ENDOFSSH here-document, so that EOF is correctly recognized as the end-marker for the nested <<EOF here-document. There are two ways to preserve the indentation:

1. Indent everything but the EOF end-marker:

   sshuserconnect user@$servername <<ENDOFSSH
   
       sqlplus -s /nolog <<EOF > /home/user/output.txt
       connect user/pwd@$servername:<port>/<DB schema>
       select column1||','||value from table where column1 like 'param1';
   EOF
   
       version=\$(grep version /home/user/output.txt | cut -f 2 -d ",")
       mkdir /home/user/\${version}
   
   ENDOFSSH
   

   This looks somewhat ugly.

2. Indent with TABs rather than spaces, and use <<-ENDOFSSH instead of <<ENDOFSSH, which causes the leading tab characters in the here document to be removed. In the code below I purposefully marked the tab characters with <TAB>, both for explicitness and because StackOverflow renders tabs with spaces, meaning that copy&pasting the code would not work correctly:

   sshuserconnect user@$servername <<-ENDOFSSH
   
   <TAB>sqlplus -s /nolog <<EOF > /home/user/output.txt
   <TAB>connect user/pwd@$servername:<port>/<DB schema>
   <TAB>select column1||','||value from table where column1 like 'param1';
   <TAB>EOF
   
   <TAB>version=\$(grep version /home/user/output.txt | cut -f 2 -d ",")
   <TAB>mkdir /home/user/\${version}
   
   ENDOFSSH