1
function readIpFile () {
configfile='node_list.txt'
    [ $# -gt 0 ] && [ -r "$1" ] && configfile="$1"

    sed -e 's/[[:space:]]*#.*// ; /^[[:space:]]*$/d' "$configfile" |
    while read target role;
    do
    case "$role" in
        m)  esMaster="\"$esMaster\", \"$target\""
            ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: false/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        k)  kbip=$target
            echo "The Kibana node IP is: $kbip"
            echo "Beginning remote kibana deployment!"
            deployRemoteKibana
            ;;
        d)  ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: false/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        i)  ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: true/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        c)  ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: false/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        di) ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: true/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        md) ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: false/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        mi) ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: false/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: true/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        mdi)ssh -n "root@$target" "sed  -i 's/#node\.master: false/node\.master: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.data: false/node\.data: true/' /etc/elasticsearch/elasticsearch.yml"
            ssh -n "root@$target" "sed  -i 's/#node\.ingest: false/node\.ingest: true/' /etc/elasticsearch/elasticsearch.yml"
            echo "Node configuration is set!"
            ssh -n "root@$target" "service elasticsearch start"
            ;;
        *)  echo "Unrecognized node setting in node_list.txt!"
            exit
    esac
    recordNodes
    deployRemoteElastic
    applyDiscoverHosts
    done
}

Here is my goal with the above code. I have a text file. The text file contains the following.

#######
#Instructions to configure file
#
#
#######

xxx.xxx.xxx.xxx m
xxx.xxx.xxx.xxx k
xxx.xxx.xxx.xxx d

The above case statement should select based on the letter which is the last field in each row in the file. This letter should be saved to the $role variable. The IP address should be saved to $target, and $target should be appended to $esMaster. Both of these variables will be used outside the while loop.

Yet every time I execute my code, $kbip is empty, and I get this error

ssh: Could not resolve hostname : Name or service not known

How can I read from my file and insert field 1, and field 2, into their respective variables, while also making those variables available outside the scope of the while loop? From what research I have done, this BASH script should not even be creating a subshell?

I am considering invoking the remote deployment functions inside the while loop, where it will make use of the variables inside the subshell? I dont want to do that though as I want to try to keep all my function calls relatively grouped together so I can move them around as needed.

I am not sure what better way there is to iterate through the file though.

Candlejack
  • 25
  • 5
  • 1
    not sure I understand `$kbip is empty` and why you think it relates to a ssh error since `kbip` is only referenced under option `k` and there is no `ssh` invocation under option `k`; as for the `ssh` error, I'd test one line of input at a time (ie, put one line of data in `node_list.txt`) and `set -x` before entering the `while` loop (`set +x` after exiting the `while` loop) and see what's generated as output ... are variables getting set as expected? what is being passed to `ssh`? are any `ssh` commands executing successfully? are you sure you're ignoring the blank/comment lines? – markp-fuso Feb 08 '20 at 15:40
  • what is `recordnodes` and have you verified it is not generating the ssh error? – markp-fuso Feb 08 '20 at 15:47
  • All targets are saved to an array in the function recordnodes. The functions purpose is to ensure that all the config changes are propagated to each host as needed. $kbip is empty, I was assuming it was because the variable is not updating as it is used outside of the loop. EDIT: Also please see updated script. I made a few changes, as I am starting to think I have no choice but to call functions from inside the loop structure. – Candlejack Feb 08 '20 at 15:51
  • 1
    You do realize `sed` can perform multiple operations during a single execution? Not that it's ever appropriate to use structure-unaware tools to edit structured data in the first place; the Right Thing is a tool like `yq`, or simply a Python YAML library. – Charles Duffy Feb 08 '20 at 16:06
  • I am aware, but as I am relatively new to serious BASH scripting I am trying to make it easy on myself (and failing, apparently). Right now my goal is to make the script run, then I can go back through on another pass and condense redundant commands as needed. EDIT: I have zero experience with python or yq. That is not an option right now as I would be starting even closer to square 1 then I already am. – Candlejack Feb 08 '20 at 16:08
  • There are already answered Q&A entries about passing arbitrary functions and variables over SSH to the remote system. – Charles Duffy Feb 08 '20 at 16:09
  • Worry more about correctness. Bash is full of hidden pitfalls; doing the obvious thing is almost always full of bugs. A good place to start is https://mywiki.wooledge.org/BashPitfalls and https://mywiki.wooledge.org/BashFAQ – Charles Duffy Feb 08 '20 at 16:10
  • I am not trying to pass the variables over SSH to the remote system, I am trying to pass variables to other functions outside of the while loop. I feel I was very clear in stating my requirements: I need to make the variables defined inside the loop available outside the loop. I was unsure if my loop was entering a subshell, which would change how I write this control structure as that would limit my options (I have no desire to make variables defined in a subshell available to the parent). – Candlejack Feb 08 '20 at 16:12
  • ...and if you care about not running a risk of silently corrupting your output file, you're better off going back to Square 1 than choosing up use tools that tend to have major footguns attached to practices that look safe at first glance. – Charles Duffy Feb 08 '20 at 16:13
  • Re; passing variables out of the loop, the issue preventing that is BashFAQ #24: http://mywiki.wooledge.org/BashFAQ/024 – Charles Duffy Feb 08 '20 at 16:13
  • ...that said, your code doesn't show you try to use variables defined in the loop after the loop is over, so I don't know why it's used in a question that's purportedly about such variables no longer being defined. – Charles Duffy Feb 08 '20 at 16:22
  • I did not show that function as that function itself works as intended. – Candlejack Feb 08 '20 at 16:23
  • ...if you want to fix that issue, so the variables are left in place after the loop finishes, move `sed -e 's/[[:space:]]*#.*// ; /^[[:space:]]*$/d' "$configfile" |` from before your `while read` to instead be `< <(sed -e 's/[[:space:]]*#.*// ; /^[[:space:]]*$/d' "$configfile")` after the `done`. – Charles Duffy Feb 08 '20 at 16:23
  • Sure, but you're expected to show a [mre] -- the shortest possible code that *demonstrates a specific problem*. That means not just showing the code that has a problem, but code that has output that demonstrably exhibits an issue ("demonstrably" so a fixed version can be *demonstrated* by someone writing an answer to now emit output that indicates that the issue at hand is fixed). – Charles Duffy Feb 08 '20 at 16:25
  • (Not requiring input files or network resources where at all avoidable is also part of making sure someone can reproduce your problem themselves, and thus test either their own proposed fixes or others' proposed answers). – Charles Duffy Feb 08 '20 at 16:28
  • There is a lot to comment on in your code, but I think the first comment, recommending `set -x` is the place to start. OH, and if somehow Windows has been involved in creating any of your files, it can hurt to `dos2unix file1 file2 ....` them to clean out any `^M` chars that might be messing things up. Agree with comments that you need to reduce this down to 1 case that fails, but you might spot the problem when you go to debugging mode with `set -x`. Good luck! – shellter Feb 08 '20 at 16:33
  • Set -X was what did it, I wasn't even entering the while loop (Probably why the variables werent being populated!). I ended up throwing out the entire CASE structure and I am just filtering my node list file into several tmp files instead with AWK. – Candlejack Feb 09 '20 at 13:27

1 Answers1

0

Your script is invoking ssh during the process, therefore all local (per function) and global (per script) variables are not passed to the called shell (ssh).

I would not use this approach personally, but would do that:

  • Read the target
  • Send a script doing whatever is required (the switch part) to the target
  • Execute the script on the target

You may also use scp to send file, and that's probably the most important part: you are not enforced to stay on the client, you may execute on the server as well - and perhaps with less privilege than root if you can afford to.

NoDataFound
  • 11,381
  • 33
  • 59
  • I am not worried about root, as this script as it is written is intended for development lab environments, and not production. Nothing sensitive will be touched by this script when accessing root. Later in the script I use heredocs to pass functions and variables to the remote shell for execution. The only problem is making the variables defined in this loop, available for the heredoc to expand client-side, so that they can then be defined server-side and used by the remote functions. – Candlejack Feb 08 '20 at 15:14
  • Then send the variable over. This work better by writing a script especially when it seems that all your ssh calls are the same for each case condition. – NoDataFound Feb 08 '20 at 15:15
  • The variable also needs to be expanded locally, as the elasticsearch node the script runs on needs to know what all the MDI nodes are, but not the kibana node. But the kibana node alsos needs access to that variable when I insert it into the config line. Basically I am trying to execute everything from one machine, using as few scripts as possible. But this is also getting a bit far afield of my original question. – Candlejack Feb 08 '20 at 15:26