5

Cannot use $$ character in environment variables of GitLab CI.

I store my secrects in GitLab CI environment variables out of which one of my pass word string has $$ characters at the end (eg :Ab1ab$$). When i echo it out or use envsubst, the output will be Ab1ab$, stripping the extra $ on the end.

I tried using to surround the string in '',"" none of which helped. Saw some concerns on the gitlab community regarding the usage of $ character in the env variables, but could not find any workarounds for my exact problem.

https://gitlab.com/gitlab-org/gitlab-ce/issues/27436

export VAR= 'Ab1ab$$' echo $VAR=Ab1ab$

maddie
  • 505
  • 1
  • 6
  • 13

1 Answers1

1

This is a pretty common problem with setting env variables in CI! I have encountered it numerous times with CircleCI as well. I have found great success with backslashes, typically, but I'm guessing you tried that already.

I found a thread discussing what I believe is likely a deeply related issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/27436

Key takeaway there is that at least in that version of the GitLab product they hadn't found a great answer yet. I decided to start doing some experimentation:

https://gitlab.com/mattbornski/gitlab-ci-exploration

I set up a whole bunch of environment variables in different formats to see which resulted in a dollar sign.

$ python3 audit_environment.py
DOLLA_DOUBLE_MIDSTRING = 1$234
DOLLA_QUAD_MIDSTRING = 1$$234
DOLLA_SINGLE_MIDSTRING_BACKSLASHED = 1\34
DOLLA_DOUBLE_MIDSTRING_BACKSLASHED = 1\\34
DOLLA_DOUBLE_MIDSTRING_LEADING_BACKSLASHED = 1\$234
DOLLA_QUAD_ENDSTRING = 1234$$
DOLLA_SINGLE_MIDSTRING_BACKSLASHED_DOUBLE_QUOTED = "1\34"
DOLLA_SINGLE_MIDSTRING_DOUBLE_QUOTED = "134"
DOLLA_SINGLE_MIDSTRING_BACKSLASHED_SINGLE_QUOTED = '1\34'
DOLLA_SINGLE_MIDSTRING = 134
Job succeeded

So, takeaway here:

  • Double dollar signs result in a single dollar sign appearing in the environment variable accessible to your code in GitLab CI. This can be repeated.
  • Quoting passes through but does not impact the expansion/elision of the dollar sign
  • Backslash passes through but does not impact the expansion/elision of the dollar sign

So your options look like:

  • Put four dollar signs in the var that you set so that you get two in the environment in which you run
  • Regenerate your keys so that they don't have problematic characters in them (a very real option that I have definitely availed myself of many times to avoid this rabbit hole)

For further exploration, here are a bunch more fun options to try: https://unix.stackexchange.com/a/309791

mattbornski
  • 11,895
  • 4
  • 31
  • 25
  • I have seen the same on-going issue with git lab and mentioned the same in the description section. I tried your 4 $$$$ suggestion already and it is stripping all the 4 and appending 1. Input : Ab1ab$$$$ Output: AB1ab1. – maddie Apr 26 '19 at 15:23
  • Unfortunately, i cannot use your second option since the secret that i am trying to use is from a different shared system across the company, which i do not have access or control to suggest changes. The only workaround i can find so far is to store it base64 encoded and decode at CI time, i know which is not pretty – maddie Apr 26 '19 at 15:26
  • @maddie it's very interesting that it results in a `1`. Are there any suspicious looking environment variables that have a dollar sign in the *name* ? – mattbornski Apr 26 '19 at 18:01
  • @maddie do you use a custom or specific runner for your CI? – mattbornski Apr 26 '19 at 18:03
  • @ mattbornski 1. My exact varibale is sth like this "6ab@sh$$" numbers and letters changed , and my output would be 6ab@sh$. When i use "6ab@sh$$$$" the output is 6ab@sh1. 2.For the second question, i use couple of shared ci runners defined by using tags in my ci, out of which the available runner picks up my job. – maddie Apr 26 '19 at 18:28