0
sed -i "s|{{PLACEHOLDER}}|${KEY_B64}|g" <path>

The command above is executed in a Gitlab CI runner and throws the following:

sed: unmatched '|'

I have double-checked the KEY_B64 environment variable, it is set and looks valid. This variable is a base-64 encoded JWT token (Kubernetes secrets expect to be base-64 encoded.

What is really strange though is that this command works fine if I run it locally (Ubuntu 22.04) and replace the env variable with the output from echo -n <JWT_TOKEN> | base64.

Based on the error message, it seems that the env value might contain the delimiter, but changing it to anything else doesn't solve the problem. On top of that, the encoded value for sure doesn't include such symbols.

What could be the cause of the issue?

Updates:

Running sed --version outputs:

$ sed --version
This is not GNU sed version 4.0

Looking with the set -x option on, I can see that the encoded string includes newlines (outputted the variable in the pipeline logs).

  • Using printf %s $VAR did not solve the issue
  • Surprisingly, base64 doesn't support -w0
Don Draper
  • 463
  • 7
  • 21
  • 3
    Use `set -x` to have an insight of what's happening – Diego Torres Milano Dec 06 '22 at 18:59
  • Add output of `echo "${KEY_B64}" | hexdump -C` to your question (no comment). – Cyrus Dec 06 '22 at 19:35
  • Include the output of `sed --version`, as there are several different versions/flavours out there. – SiKing Dec 06 '22 at 23:16
  • The question needs to include enough details to let us reproduce the problem ourselves. Starting with `set -x` logs, and then modifying the logged command to no longer have confidential information while ensuring that it still produces the problem, is the best way to get there. – Charles Duffy Dec 06 '22 at 23:22
  • BTW, `echo -n` is itself nonportable/unreliable; [the POSIX specification for `echo`](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html) explicitly recommends using `printf` instead (in this case `printf %s "$foo"` instead of `echo -n "$foo"` -- note that when using printf correctly, the data stays out-of-band from the format string) if you might use either `-n` or escape sequences. See also the [unix.se] question [Why is printf better than echo?](https://unix.stackexchange.com/a/65819/3113) – Charles Duffy Dec 06 '22 at 23:24
  • One other thing: If you don't pass `-w 0` to base64, some versions will line wrap by default (it's implementation-specific whether that's default only when output is to a TTY or _always_ default behavior). If you've got newlines in your `sed` expression, that can _definitely_ shake things up; though it's another thing that will be there in the `set -x` logs, so they're really the right place to start. – Charles Duffy Dec 06 '22 at 23:25
  • @CharlesDuffy, check the updates, please – Don Draper Dec 07 '22 at 06:42
  • @SiKing, I have added more details – Don Draper Dec 07 '22 at 06:42
  • `-w` is present in the GNU coreutils version, but if this is an Alpine container or such, it might be using busybox – Charles Duffy Dec 07 '22 at 18:09
  • "This is not GNU sed version 4.0": so the `-i` *might* require a value; see `man sed` on your system. You can pass a blank value with `-i ''` if you do not want the backup. I have been caught by this many times, especially on Macs. – SiKing Dec 09 '22 at 20:13

1 Answers1

0

Using set -x revealed that for some reason the base64 command added line breaks to the output.

As base64 -w0 is not supported, I had to use the command below to remove the newlines in the base64 output, which solved the problem.

export MY_KEY_B64=$(echo -n $MY_KEY | base64 | tr -d \\n)

Note: as I was fairly told in the comments, using printf is preferable but in this case, it was not the cause of the issue so I did not modify this command to emphasize what cause the problem.

Don Draper
  • 463
  • 7
  • 21
  • At least add quotes -- `echo -n "$MY_KEY"`. You don't want a whitespace-surrounded `*` in your key to be converted to a list of filenames. (Also see [I just assigned a variable, but `echo $variable` shows something different!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else)) – Charles Duffy Dec 07 '22 at 18:08