How do I remove the EOL/^M/Ctrl-M from inside a bash script?

Question

Problem: I need to remove the EOL/^M/Ctrl-M which comes from writing a document in Windows/Dos and opening it in Unix.

Objective: I need to read a file and assign the read string to a variable, so that I can use the small string as part of a path name. I've read many posts about opening VIM, however, this needs to be done via a SOURCED BASH SCRIPT, will be run on multiple machines, and has many environment variables. All the code needs to be native, therefore, no dos2unix or something similar. I've also tried several lines of code for sed, perl, and awk; however, I haven't had much luck. Any help would be great.
Example of executing the sourced script from the terminal:

     username#: . ./Do_something.sh  some_string > ~/Do_something_log.txt

Let's pretend we have the following lines inside Do_something.sh

 #!/bin/bash
 some_string=$1 
 SOME_PATH="${ANOTHER_PATH}/Some/directory_name"
 FILE="${SOME_PATH}/an/even/longer/path/a_file.cfg"

 while read line
 do
      if [ ! -z "{line}" ] ; then
         NEW_PATH_VARIABLE=${line}
         TOTAL_PATH="${SOME_PATH}/${NEW_PATH_VARIABLE}.cfg"
         if [ ! -f "${TOTAL_PATH}" ] ; then
             echo "ERROR. FILE NOT FOUND!!!"
         fi
         if [ "${SOME_STRING}" = "${TOTAL_PATH}" ] ; then
             break
         fi
       fi
 done < ${ANOTHER_FILE}

After reading the file, I have an EOL inside my variable name, so it is unable to tag the .cfg extension on the end of the variable which is being assigned to TOTAL_PATH. I only noticed the ^M because I executed less from the terminal using

    username#: less ~/Do_something_log.txt

I am looking for the most efficient way to remove remove these (EOL/^M/Ctrl-M)s from inside a script.

Thanks

http://stackoverflow.com/questions/800030/remove-carriage-return-in-unix — RobEarl, Aug 25 '14 at 10:21
Did you try it with `\r`? http://en.wikipedia.org/wiki/Carriage_return — RobEarl, Aug 25 '14 at 10:46
Sorry, had a problem with that last post. Neither `tr -d '^M' < infile > outfile` nor `tr -d '\r' < infile > outfile` worked (after adjusting the following code, of course). In addition to that, this needs to be as compatible as possible. cat doesn't seem to be available on macs. — David, Aug 25 '14 at 10:47
`tr -d '\015' newfile` worked for me on OSX. It's not like there's not thousands of duplicates of this question already. — tripleee, Aug 25 '14 at 10:52
Sweet! Thanks @tripleee. I hadn't thought about submitting the ascii code. And yes there are indeed thousands which makes it ever more frustrating when you can't figure it out. :/ — David, Aug 25 '14 at 11:03

Matt · Answer 1 · 2014-08-25T10:53:03.960

1

Use Bash substring replacement,

${var/%find/replace}

The % requires the find value to be at the end of the string.

NEW_PATH_VARIABLE=${line/%$'\r'/}

The $'\r' search can be replaced with the Ctrl-V Ctrl-M sequence if needed.

edited Aug 25 '14 at 10:53

answered Aug 25 '14 at 10:46

Matt

68,711
7
155
158

Maybe I missed something. But the character which I am trying to remove is ^M. Why do you recommend '\r'? – David Aug 25 '14 at 10:49
So, I'm using a mac to program this script. Any idea how to get that Ctrl-V Ctrl-M sequence-character? – David Aug 25 '14 at 10:55
In a terminal window its ctrl-v ctrl-m. I would recommend using the string `$'\r'` though so you don't get stuck with control characters. – Matt Aug 25 '14 at 10:57
In Terminal on a Mac, it seems that ctrl-V ctrl-M simply inserts a newline. )-: – tripleee Aug 25 '14 at 11:00
hmmm.. not on mine!? What shell are you using? I'm Bash. – Matt Aug 25 '14 at 11:01
Windows standard EOL = `CRLF` (2 bytes, a.k.a. `\r\n`, 0x0d0a, Ctrl-M+Ctrl-J); MAC standard EOL = `CR` (1 byte, a.k.a. `\r`, 0x0d, Ctrl-M); UNIX standard EOL = `LF` (1 byte, a.k.a. `\n`, 0x0a, Ctrl-J)...this is the result of committees... – skeetastax Feb 23 '21 at 00:09

How do I remove the EOL/^M/Ctrl-M from inside a bash script?

1 Answers1