Why does curl command on windows generate a /bin/bash: line /: $'\r' : command not found error?

Question

I am trying to install Operator Lifecycle Manager (OLM) — a tool to help manage the Operators running on your cluster — from the official documentation, but I keep getting the error below. What could possibly be wrong?

This is the result from the command:

curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/install.sh | bash -s v0.22.0

/bin/bash: line 2: $'\r': command not found
/bin/bash: line 3: $'\r': command not found
/bin/bash: line 5: $'\r': command not found
: invalid option6: set: -
set: usage: set [-abefhkmnptuvxBCEHPT] [-o option-name] [--] [-] [arg ...]
/bin/bash: line 7: $'\r': command not found
/bin/bash: line 9: $'\r': command not found
/bin/bash: line 60: syntax error: unexpected end of file

I've tried removing the existing curl and downloaded and installed another version but the issue has still persisted. Most solutions online are for Linux users and they all lead to Windows path settings and files issues.

I haven't found one tackling installing a file using curl.

I'll gladly accept any help.

Answer 1

Using PowerShell on Windows, you must explicitly ensure that the stdout lines emitted by curl.exe are separated with Unix-format LF-only newlines, \n, when PowerShell passes them on to bash, given that bash, like other Unix shells, doesn't recognize Windows-format CRLF newlines, \r\n:

The simplest way to avoid the problem is to call via cmd /c:

cmd /c 'curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/install.sh | bash -s v0.22.0'

cmd.exe's pipeline (|) (as well as its redirection operator, >), unlike PowerShell's (see below), acts as a raw byte conduit, so it simply streams whatever bytes curl.exe outputs to the receiving bash call, unaltered.

Fixing the problem on the PowerShell side requires more work, and is inherently slower:

(
  (
    curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/install.sh 
  ) -join "`n"
) + "`n" | bash -s v0.22.0

^{Note: `n is a PowerShell escape sequence that produces a literal LF character, analogous to \n in certain bash contexts.}

Note:

• It is important to note that, as of PowerShell 7.2.x, passing raw bytes through the pipeline is not supported: external-program stdout output is invariably decoded into .NET strings on reading, and re-encoded based on the $OutputEncoding preference variable when writing to an(other) external program.

  • See this answer for more information, and GitHub issue #1908 for potential future support for raw byte streaming between external programs and on redirection to a file.

• That is, PowerShell invariably interprets output from external programs, such as curl.exe, as text, and sends it line by line through the pipeline, as .NET string objects (the PowerShell pipeline in general conducts (.NET) objects).

  • Note that these lines (strings) do not have a trailing newline themselves; that is, the information about what specific newline sequences originally separated the lines is lost at that point (PowerShell itself recognizes CRLF and LF newlines interchangeably).

• However, if the receiving command is also an external program, PowerShell adds a trailing platform-native newline to each line, which on Windows is a CRLF newline - this is what caused the problem.

• By collecting the lines in an array up front, using (...), they can be sent as a single, LF-separated multi-line string, using the -joinoperator, as shown above.

  • Note that PowerShell appends a trailing platform-native newline to this single, multi-line string too, but a stray \r\n at the very end of the input is in effect ignored by bash, assuming that the last true input line ends in \n, which is what the extra + "`n" at the end of the expression ensures.

  • However, there are scenarios where this trailing CRLF newline does cause problems - see this answer for an example and workarounds via the platform-native shell.

Answer 2

To start off, I need to be clear about a few things:

1. Based on the tags to the question, I see we are in PowerShell rather than a linux/unix or even Windows cmd shell
2. In spite of this, we are using Unix curl (probably curl.exe), and not the PowerShell alias for Invoke-WebRequest. We know this because of the -sL argument. If Powershell was using the alias, we'd see a completely different error.

Next, I need to talk briefly about line endings. Instead of just a single LF (\n) character as seen in Unix/linux and expected by bash, Windows by default uses the two-character LF/CR pair (\n\r) for line endings.

---

With all that background out of the way, I can now explain what's causing the problem. It's this single pipe character:

| 

This a PowerShell pipe, not a Unix pipe, so the operation puts the output of the curl program in the PowerShell pipeline in order to send it to the bash interpreter. Each line is an individual item on the pipeline, and as such no longer includes any original line breaks. PowerShell pipeline will "correct" this before calling bash using the default line ending for the system, which in this case is the LF/CR pair used by Windows. Now when bash tries to interpret the input, it sees an extra \r character after every line and doesn't know what to do with it.

The trick is most of what we might do in Powershell to strip out those extra characters is still gonna get sent through another pipe after we're done. I guess we could tell curl to write the file to disk without ever using a pipe, and then tell bash to run the saved file, but that's awkward, extra work, and much slower.

---

But we can do a little better. PowerShell by default treats each line returned by curl as a separate item on the pipeline. We can "trick" it to instead putting one big item on the pipeline using the -join operation. That will give us one big string that can go on the pipeline as a single element. It will still end up with an extra \r character, but by the time bash sees it the script will have done it's work.

Code to make this work is found in the other answer, and they deserve all the credit for the solution. The purpose of my post is to do a little better job explaining what's going on: why we have a problem, and why the solution works, since I had to read through that answer a couple times to really get it.