0

The issue I'm having is that the DB_PASS variable has an extra unwanted space \n and I'm not sure where it is coming from. I'm assuming I made a mistake in my shell script?

This is a code snippet of my shell script that replaces all the variables with their corresponding values:

#!/usr/bin/env bash

sed "s/GCLOUD_SERVICE_KEY_ENC_REPLACE/`echo $GCLOUD_SERVICE_KEY_ENC|base64`/g" mlflow_gcp_secret.yaml | \
sed "s/GCP_STORAGE_BUCKET_REPLACE/`echo $GCP_STORAGE_BUCKET|base64`/g" | \
sed "s/CLOUDSQL_USER_REPLACE/`echo $DB_USER|base64`/g" | \
sed "s/CLOUDSQL_PASS_REPLACE/`echo $DB_PASS|base64`/g" | \
sed "s/CLOUDSQL_DB_REPLACE/`echo $DB_NAME|base64`/g" | \
kubectl apply -f -

I would run this shell script as follows: 

export GCLOUD_SERVICE_KEY_ENC=$(cat <PATH_TO_SECRET_JSON> | base64)
export GCP_STORAGE_BUCKET=<GS_BUCKET_NAME> # exclude gs:// prefix
export DB_USER=<DB_USER>
export DB_PASS=<DB_PASS>
export DB_NAME=<DB_NAME>
source ./populate_secret.sh
Riley Hun
  • 2,541
  • 5
  • 31
  • 77
  • 1
    `DB_USER = ` bash doesn't allow a space surrounding the `'='`. the rest have the same problem. – David C. Rankin Jan 14 '20 at 00:39
  • Ok thanks - but still doesn't resolve the issue – Riley Hun Jan 14 '20 at 00:48
  • 2
    `echo` appends a newline to whatever it prints; use e.g. `printf "%s" "$DB_PASS" | base64` instead (and similarly for all the others). Also, put double-quotes around all variable references (as in my example). Finally, I recommend `$( )` instead of backticks. So: `sed "s/CLOUDSQL_USER_REPLACE/$(printf "%s" "$DB_USER"|base64)/g"` – Gordon Davisson Jan 14 '20 at 00:51
  • As the code is for deployment, hopefully you won’t mind a couple of extra suggestions: I agree with @GordonDavisson on trading in `echo` for `printf` (more portable, more robust), but I would also trade in `printf` for a here-string. `printf` isn’t doing anything functional here, it’s just escorting a string to `base64`). A here-string does this without an program; yes, `printf` is a builtin, but is replaced by an external program on some systems (but not so commonly as `echo`): `$(printf "$DB_FOO") | base64)` will become `$(base64<<<"$DB_FOO")`. _Cont’d…_ – CJK Jan 14 '20 at 14:48
  • _…Cont’d_ `cat` often gets employed needlessly in a lot of scripts in the exact way you’ve done so with `$(cat | base64)`. Like above, it doesn’t really **do** anything that `base64` can’t do itself. Consider instead: `"$(base64 "$JSONPATH")"` (and don’t forget your quotes). Finally, the obsession with `sed` everyone seems to have can make it easy to overlook other options. `sed` is a very powerful tool, but not being used here to do anything `bash` itself can’t do, probably quicker and with one fewer dependency: _Cont’d…_ – CJK Jan 14 '20 at 15:24
  • _…Cont’d_ All of your string replacement operations look to be using fixed strings, where the `sed` command runs the substitution program `"s/$STRING1/$STRING2/g"` that will replace all occurrences of `$STRING2` that appear in the supplied input with the value of `$STRING1`. That is what `bash` parameter expansions can do, and takes the form `"${input//$STRING1/$STRING2}"`. You’ll see the syntax is similar, but the text to be altered is first assigned to a shell variable (in this case, I’ve assumed it’s stored in `$input`). _Cont’d…_ – CJK Jan 14 '20 at 15:42
  • _…Cont’d_ You won’t be able to chain your replacements together as with `sed`, as the returned value from `output="${input//…/…}”` needs to be assigned to a variable before it can be processed again (because notice that `input` _appears_ to lack a `$`, so might look like an explicit value, which means that explicit values can’t be used in the first parameter). But all of these suggestions save on creating extra shell sub-processes, and I, personally, think breaking the string replacements up will make the script easier to read as well. – CJK Jan 14 '20 at 15:48
  • Really helpful suggestions! Thanks a million! – Riley Hun Jan 14 '20 at 18:20
  • @CJK Unfortunately `<<<` also appends a newline, so it won't work here. Also, since the YAML that all the replacements are being applied to is in a file rather than a variable I'd stick with `sed`. If you're concerned about efficiency, though, you can use a single instance of `sed` and use the `-e` option to pass it a bunch of replacements (`sed -e "s/thing1/repl1/g" -e "s/thing2/repl2/g" -e ...`). – Gordon Davisson Jan 14 '20 at 19:11
  • @GordonDavisson, you're quite right, I forgot about the newline. Thanks for catching that. – CJK Jan 30 '20 at 04:23

0 Answers0