0

I have a bitbucket pipelines config which builds my app.yaml file by replacing matching strings with pipeline environment variables. Some of these variables have special characters in and cause SED to fail. How can I get around this?

My pipelines script is this:

script:
          - cp app.yaml.dist app.yaml
          - cat app.yaml
          - echo ${APP_GOOGLE_CLIENT_SECRET} > src/config/credentials/gcloud.json

          - sed -i "s/\$DEBUG/$DEBUG/" app.yaml
          - sed -i "s/\$DATABASE_URL/$STAGING_MONGOURL/" app.yaml

So I create an app.yaml file from my dist version. Then I replace the variables in my app.yaml file such as $DEBUG and $DATABASE_URL with environment variables which are stored in pipelines.

If it helps my app.yaml file environment variables section is basically this:

...
env_variables:
  APP_DEBUG: $DEBUG
  DATABASE_URL: $DATABASE_URL
...

The error I get is sed: -e expression #1, char 35: unknown option to 's'

Thanks in advance

Stretch0
  • 8,362
  • 13
  • 71
  • 133
  • \`$STAGING_MONGOURL\` <-- back tick ? intentional – asio_guy Nov 02 '17 at 10:11
  • Sorry that was left over from me testing something. I'm not actually using them. Will update it – Stretch0 Nov 02 '17 at 10:16
  • see https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed , https://unix.stackexchange.com/questions/129059/how-to-ensure-that-string-interpolated-into-sed-substitution-escapes-all-metac and https://unix.stackexchange.com/questions/32907/what-characters-do-i-need-to-escape-when-using-sed-in-a-sh-script – Sundeep Nov 02 '17 at 10:41
  • [Check this](https://stackoverflow.com/questions/24705650/sed-unknown-option-to-s-in-bash-script) – mathB Nov 02 '17 at 10:42
  • Was just having a look at them. Trying to use `- sed -i "s/\$DATABASE_URL/$(echo $STAGING_MONGOURL | sed -e 's/[\/&]/\\&/g'" app.yaml` but getting error line 81: unexpected EOF while looking for matching `"'. Not sure if it's not escaping properly or what. There definitely isn't any trailing quotes of any sort – Stretch0 Nov 02 '17 at 10:49
  • You said "Some of these variables have special characters in". Use the one that is **not** one of those special characters. Example, if your variables doesn't have `:`, then use it. `sed -i "s:\$DEBUG:$DEBUG:" app.yaml` – mathB Nov 02 '17 at 10:58
  • $DATABASE_URL is a url and contains a lot of special chars. i.e. `: / - = . , ? &` – Stretch0 Nov 02 '17 at 11:20
  • 1
    how about using `@` then? `sed -i "s@\$DEBUG@$DEBUG@" app.yaml` Or `#` or `!` – mathB Nov 02 '17 at 11:35
  • Ended up going down that path. Got it working by using `- sed -i "s|\$DATABASE_URL|$STAGING_MONGOURL|" app.yaml` – Stretch0 Nov 02 '17 at 12:39
  • "...and contains a lot of special chars...." It seems clear that people missed that info in your title. Next time, best to include some sample data in your Q, so it is copy/pastable to a readers command-line ;-) Good luck. – shellter Nov 02 '17 at 13:02

1 Answers1

1

As discussed in the comments, you need to use one of the special characters that is not in your variable.

sed -i "s|\$DATABASE_URL|$STAGING_MONGOURL|" app.yaml
sed -i "s|\$DEBUG|$DEBUG|" app.yaml
mathB
  • 634
  • 8
  • 21