Using echo in bash puts last variable in front of the output

Question

I'm trying to write a script and one of the parts of the script requires me to concatenate some variables together to create a URL.

REPO_URL='https://github.com/Example/Repo.Game/'
FILENAME='Example.Game-linux.zip'
latest_version="$(curl -LIs "${REPO_URL}/releases/latest" | grep -i '^location:' | cut -d' ' -f2 | cut -d'/' -f8)"

echo "$latest_version"
echo "$FILENAME"
echo "$REPO_URL"
echo "${REPO_URL}releases/download/${latest_version}/${FILENAME}"

Output:

2.0.5164
Example.Game-linux.zip
https://github.com/Example/Repo.Game/
/Example.Game-linux.ziple/Repo.Game/releases/download/2.0.5164

My actual output:

2.0.5164
Oxide.Rust-linux.zip
https://github.com/OxideMod/Oxide.Rust/
/Oxide.Rust-linux.zipideMod/Oxide.Rust/releases/download/2.0.5164

It looks like some kind of overflow problem? I'm not exactly sure. I added abcabc to the filename and the output became

/Oxide.Rust-linux.zipabcabc/Oxide.Rust/releases/download/2.0.5164

Any help would be appreciated.

Hello DrSulfurious, warning with GitHub.com output. All outputs are with CRLF (\r\n) at end of line like under Windows. Not single LF (\n). You can see them with a "`| cat -A`" after you script command line execution. "\r" are printed like "`^M`". So put a "`| dos2unix`" between "`curl`" and "`grep`" — Arnaud Valmary, Aug 21 '21 at 17:43
@ArnaudValmary that's very insightful. Thank you for that advice. — Skullfurious, Aug 21 '21 at 17:55
GitHub has an API go collect this information reliably: https://docs.github.com/en/rest — Léa Gris, Aug 21 '21 at 18:10

score 0 · Answer 1 · answered Aug 21 '21 at 17:49

0

I resolved the problem by removing the carriage return value from the variable.

tr -d '\r' seems to have resolved it. I'm not sure where the variable came from and if anyone has advice on how to clean up this mess I would love some advice.

latest_version="$(curl -LIs "${REPO_URL}/releases/latest" | grep -i '^location:' | cut -d' ' -f2 | cut -d'/' -f8 | tr -d '\r')

answered Aug 21 '21 at 17:49

Skullfurious

1
1

Internet protocols (including http) generally follow the DOS/Windows convention that uses CR+LF line endings. See ["Are shell scripts sensitive to encoding and line endings?"](https://stackoverflow.com/questions/39527571/are-shell-scripts-sensitive-to-encoding-and-line-endings) – Gordon Davisson Aug 21 '21 at 19:31

score 0 · Answer 2 · answered Aug 21 '21 at 21:33

You can use ANSI quoting, and variable substitution to remove control characters from variables without having to invoke sub-shells.

ANSI quoting uses the special format $'\*' to represent special characters. For example use $'\t' for tab, $'\n' for new-line and $'\r' for carriage-return.

Variable substitution uses extra characters at the end of the variable name to perform actions on the variable. For example

${variable//[pattern]/[substitution]} will replace all instances of [pattern] in ${variable} with [substitution].
${variable%[pattern]} will remove [pattern] from ${variable} if it is at the end.

By combining these two, you can remove carriage-return characters from the end of your variable like this:

echo ${variable%$'\r'}

Note: Variable substitution doesn't actually change the contents of the variable. To do that, you have to re-assign the result back to the variable:

variable="${variable%$'\r'}"

score 0 · Answer 3 · answered Aug 21 '21 at 22:21

There is a cleaner way to get the version number, minus any trailing carriage-return, from github using sed.

latest_version =$(curl -LIs "${REPO_URL}/releases/latest" | sed -n 's/^Location:.*\/\([^\r]*\).*$/\1/p')

sed reads every line of input (STDIN by default) and performs operations on it defined by the action string parameter. The action string is a little tricky to explain in this case, but here goes:

The -n option suppresses the printing of each input line. Output will then only happen if it is explicitly stated in the action string.
The s/[pattern]/[substitution]/p construct says whenever you find [pattern], replace it with [substitution] and print it. Our [pattern] is ^Location:.*\/$.*$$, and our [substitution] is \1.
The expression ^ matches the beginning of the line.
The expression . means any single character, and the expression .* means any number of characters (including zero). This will match the largest possible string, so, for example .*/ will match abc/def/ in the string abc/def/ghi.
The expression \/ just escapes the forward slash (because we are using backslash as a delimiter, we have to escape it).
The expression $[pattern]$ says any time you find [pattern], remember it. in our case, it will remember whatever matches [^\r].
The expression [{chars}] matches any one of the characters in {chars}. [^{chars}] matches any character that is not in {chars}. so [^\r]* matches any number of characters that is not a carriage return.
The expression $ matches the end of a line.
The expression \1 is replaced by the first remembered pattern.

So altogether, our action string says:

If you find a line that starts with Location:, followed by any number of characters, followed by a /, followed by any number of characters that are not a carriage return (which will be remembered), followed by any number of characters, followed by an end of line, then print the remembered characters.

Using echo in bash puts last variable in front of the output

3 Answers3