How to remove a newline from a string in Bash

Question

I have the following variable.

echo "|$COMMAND|"

which returns

|
REBOOT|

How can I remove that first newline?

Answer 1

In string, character replacement / deletion

Under bash, there are some bashisms:

The tr command could be replaced by ${parameter/pattern/string} bashism:

COMMAND=$'\nREBOOT\r   \n'
echo "|${COMMAND}|"
|
   OOT
|

echo "|${COMMAND//[$'\t\r\n']}|"
|REBOOT   |

echo "|${COMMAND//[$'\t\r\n ']}|"
|REBOOT|

See Parameter Expansion and QUOTING in bash's man page:

man -Pless\ +/parameter/pattern/string bash

man -Pless\ +/\/pattern bash
man -Pless\ +/\\\'string\\\' bash

man -Pless\ +/^\\\ *Parameter\\\ Exp bash
man -Pless\ +/^\\\ *QUOTING bash

${parameter//pattern/string}
    If there are two slashes separating parameter and pattern,
    all  matches of pattern are replaced with string.

Further...

As asked by @AlexJordan, this will suppress all specified characters. So what if $COMMAND do contain spaces...

COMMAND=$'         \n        RE BOOT      \r           \n'
echo "|$COMMAND|"
|
           BOOT      
|

read -r COMMAND <<<"${COMMAND//[$'\t\r\n']}"
echo "|$COMMAND|"
|RE BOOT|

Explanation

Answering Vulwsztyn's question:

Why does this work when the pattern is empty?

In ${COMMAND//[$'\t\r\n ']} :

• The 1st slashe / mean: Pattern substitution
• The pattern is /[$'\r\n '], begin with / then all matches of pattern are replaced with string
• Then, the string is empty.

Avoid fork to tr for single string!

Let compare:

COMMAND=$'\nREBOOT\r   \n'
echo ${COMMAND@Q}
$'\nREBOOT\r \n'

COMMAND=$(echo $COMMAND|tr -d '\n\t\r ')
echo ${COMMAND@Q}
'REBOOT'

Then

time for i in {1..1000};do
    COMMAND=$'\nREBOOT\r   \n'
    COMMAND=$(echo $COMMAND|tr -d '\n\t\r ')
done;echo ${COMMAND@Q}

real    0m2.785s
user    0m2.296s
sys     0m0.774s
'REBOOT'

With

COMMAND=$'\nREBOOT\r   \n'
COMMAND="${COMMAND//[$'\t\r\n ']}"
echo ${COMMAND@Q}

time for i in {1..1000};do
    COMMAND=$'\nREBOOT\r   \n'
    COMMAND="${COMMAND//[$'\t\r\n ']}"
done;echo ${COMMAND@Q}

real    0m0.006s
user    0m0.001s
sys     0m0.004s
'REBOOT'

Doing 1'000 forks to tr take more than 2700ms on my host, while same job is done in 6ms ( 464.2x faster!! ) by using built-in bash Parameter Expansion!!

Answer 2

Clean your variable by removing all the linefeeds:

COMMAND=$(echo $COMMAND|tr -d '\n')

Answer 3

echo "|$COMMAND|"|tr '\n' ' '

will replace the newline (in POSIX/Unix it's not a carriage return) with a space.

To be honest I would think about switching away from bash to something more sane though. Or avoiding generating this malformed data in the first place.

Hmmm, this seems like it could be a horrible security hole as well, depending on where the data is coming from.

Answer 4

Using bash:

echo "|${COMMAND/$'\n'}|"

^{(Note that the control character in this question is a 'newline' (\n), not a carriage return (\r); the latter would have output REBOOT| on a single line.)}

Explanation

Uses the Bash Shell Parameter Expansion ${parameter/pattern/string}:

The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. [...] If string is null, matches of pattern are deleted and the / following pattern may be omitted.

Also uses the $'' ANSI-C quoting construct to specify a newline as $'\n'. Using a newline directly would work as well, though less pretty:

echo "|${COMMAND/
}|"

Full example

#!/bin/bash
COMMAND="$'\n'REBOOT"
echo "|${COMMAND/$'\n'}|"
# Outputs |REBOOT|

Or, using newlines:

#!/bin/bash
COMMAND="
REBOOT"
echo "|${COMMAND/
}|"
# Outputs |REBOOT|

Answer 5

Adding answer to show example of stripping multiple characters including \r using tr and using sed. And illustrating using hexdump.

In my case I had found that a command ending with awk print of the last item |awk '{print $2}' in the line included a carriage-return \r as well as quotes.

I used sed 's/["\n\r]//g' to strip both the carriage-return and quotes.

I could also have used tr -d '"\r\n'.

Interesting to note sed -z is needed if one wishes to remove \n line-feed chars.

$ COMMAND=$'\n"REBOOT"\r   \n'

$ echo "$COMMAND" |hexdump -C
00000000  0a 22 52 45 42 4f 4f 54  22 0d 20 20 20 0a 0a     |."REBOOT".   ..|

$ echo "$COMMAND" |tr -d '"\r\n' |hexdump -C
00000000  52 45 42 4f 4f 54 20 20  20                       |REBOOT   |

$ echo "$COMMAND" |sed 's/["\n\r]//g' |hexdump -C
00000000  0a 52 45 42 4f 4f 54 20  20 20 0a 0a              |.REBOOT   ..|

$ echo "$COMMAND" |sed -z 's/["\n\r]//g' |hexdump -C
00000000  52 45 42 4f 4f 54 20 20  20                       |REBOOT   |

And this is relevant: What are carriage return, linefeed, and form feed?

• CR == \r == 0x0d
• LF == \n == 0x0a

Answer 6

What worked for me was echo $testVar | tr "\n" " "

Where testVar contained my variable/script-output

Answer 7

If you are using bash with the extglob option enabled, you can remove just the trailing whitespace via:

shopt -s extglob
COMMAND=$'\nRE BOOT\r   \n'
echo "|${COMMAND%%*([$'\t\r\n '])}|"

This outputs:

|
RE BOOT|

Or replace %% with ## to replace just the leading whitespace.

Answer 8

You can simply use echo -n "|$COMMAND|".

$ man echo

-n do not output the trailing newline

Answer 9

To address one possible root of the actual issue, there is a chance you are sourcing a crlf file.

CRLF Example:

.env (crlf)

VARIABLE_A="abc"
VARIABLE_B="def"

run.sh

#!/bin/bash
source .env
echo "$VARIABLE_A"
echo "$VARIABLE_B"
echo "$VARIABLE_A $VARIABLE_B"

Returns:

abc
def
 def

If however you convert to LF:

.env (lf)

VARIABLE_A="abc"
VARIABLE_B="def"

run.sh

#!/bin/bash
source .env
echo "$VARIABLE_A"
echo "$VARIABLE_B"
echo "$VARIABLE_A $VARIABLE_B"

Returns:

abc
def
abc def

Answer 10

Use this bashism if you don't want to spawn processes like (tr, sed or awk) for such a simple task. Bash can do that alone:

COMMAND=${COMMAND//$'\n'/}

From the documentation:

${FOO//from/to} Replace all
${FOO/from/to}  Replace first match

Answer 11

Note that the shell already does that for you if you pass $COMMAND as a parameter instead of a string.

So you can try this:

COMMAND="a
   b
   c"

echo $COMMAND

To add the pipes, you can just use the quotes and no spaces like so:

echo "|"$COMMAND"|"

works just fine and it's a tad simpler than the other solutions. It also works with just /bin/sh.

Answer 12

Grep can be used to filter out any blank lines. This is useful if you have several lines of data and want to remove any empty.

echo "$COMMAND" | grep .