7

On linux, using the bash shell, when I use the script command, the generated file is called typescript. When I open that file with vim, each line contains the ^M character, and several lines (due to my colored command prompt) contain a character ^[. I would like to replace these characters with nothing, effectively removing them from the generated script.

First, I tried :%s/^[//gc, :%s/\^[//gc, :%s/\^\[//gc, and a few other variants. None of them matched the ^[ character, so the search/replace didn't work.

I also tried all these variants for the ^M character with the same results. After some googling I discovered that the ^M character is really the carriage return "\r". So then I tried :%s/\\r//gc and this worked for the ^M character!

I googled some more to try and figure out what the ^[ character is but have found nothing helpful.

2 questions:

1) What is the ^[ character, and what is the appropriate regex to use in vim to search and replace it?

2) Why, when using the script command on linux, does the generated script produce ^M at the end of the line? This makes me think the linux script command is generating CRLF eol characters rather than just LF eol characters.

axiopisty
  • 4,972
  • 8
  • 44
  • 73
  • Try using `\^\[` or Ctrl+v`[` – Explosion Pills Nov 06 '13 at 18:22
  • Is this any different than what I posted in my question: :%s/\^\\[//gc? – axiopisty Nov 06 '13 at 18:23
  • You didn't escape the bracket – Explosion Pills Nov 06 '13 at 18:23
  • I think I did. It's the 3rd example of what I tried in the OP `:%s/\^\[//gc`. It doesn't work. I think the real problem is that whatever the **^[** character is, its one character, but the search string is treating it as 2. This is the same problem I initially had with the **^M** character until I discovered it is just 1 character, the carriage return. Its kind of funny that you need to search for **\r** to replace **^M**. – axiopisty Nov 06 '13 at 18:27
  • 2
    @axiopisty LF control character tells terminal to move cursor one line down *without altering column it is on*. CR tells terminal to move cursor to the first column. Both are needed to form a new line. CRLF reduced to LF or CR is just a convention. You can check what you will see on terminals not supporting translation of LF to CRLF by using `stty -onlcr` (undo using `stty onlcr`): this will be stairs. – ZyX Nov 06 '13 at 19:38
  • 2
    By the way, `^[` is `\e` in a pattern. Though you may as well use `CTRL-V` as suggested below. – ZyX Nov 06 '13 at 19:40

5 Answers5

6

^M and ^[ are control characters. As you already pointed out correctly, they are one character, not two, you can type them in vim by pressing Ctrl+V and then Ctrl+[ to get ^[.

So the replace command you're looking for would look like s/^[//gc, with the only difference from what you've tried that you cannot type ^[ literally.

^M is a CR (carriage return character). There are commands like dos2unix to get rid of such characters. Also vim has some built in functions to get rid of them.

^[ on the other hand, is a color control character. In bash you probably get a colored output, in vim you only see the control character.

Indeed I see the same control characters when using script. Others have pointed out that this behavior is expected, I couldn't find a straight forward way to circumvent it, so I wrote a wrapper script:

#!/usr/bin/env bash

### Set the variable typescript to the last positional parameter passed to script
typescript="${!#}"
### If the last positional parameter is an option (and starts with "-"),
### set typescript to "typescript" (standard argument of script)
if [[ "${!#:0:1}" == "-" ]]; then
    typescript="typescript"
fi
### Invoke /usr/bin/script with all options passed to the wrapper script
/usr/bin/script $@
### Once script has finished, call dos2unix to get rid of control characters
dos2unix "$typescript"

Write these lines into a file called script and put it in a directory which is in the $PATH variable before /usr/bin (in my case that's ~/bin). If you now type type script, it should point you to your wrapper script, not to /usr/bin/script. When you now type script, it will invoke the wrapper script which in turn calls /usr/bin/script and dos2unix.

pfnuesel
  • 14,093
  • 14
  • 58
  • 71
  • This works to replace the **^[** character. It is useful because it helps me easily replace the character with vim. If you can modify your answer to include the other part of my question, about why the script command is generating the CRLF line endings rather than just the LF line endings, then I'll accept the answer. – axiopisty Nov 06 '13 at 18:43
  • Can you share the bash script you're using? – pfnuesel Nov 06 '13 at 18:46
  • Do you mean you want to see the file that is being generated as output from the linux `script` command? I'm not running a bash script. I'm just typing commands in a bash shell, on linux mint 15, so they will be saved to a file. The `script` command generates this file. I still have no idea why, when doing this on linux, the generated file contains CRLF line endings. – axiopisty Nov 06 '13 at 18:49
  • Sorry, I misunderstood at first. Indeed I have the same control characters. – pfnuesel Nov 06 '13 at 18:59
  • I couldn't find an easy solution for this, so I added a workaround. – pfnuesel Nov 07 '13 at 12:37
5

Why, when using the script command on linux, does the generated script produce ^M at the end of the line. This makes me think the linux script command is generating CRLF eol characters rather than just LF eol characters.

Because that's what the terminal driver inserts:

It is the terminal driver in canonical mode, "inside" the pseudo-terminal, that is expanding NLs … into CRNL pairs.

NULL
  • 2,549
  • 1
  • 15
  • 17
1

I have found that some files are written for different line endings. Unix, Dos, and Mac. you can change the way VIM see's these by re-editing the file in the following file format. I found that ^M gets changed to newline characters when editing in the mac format, so for that run this in VIM. This is not really a search and replace however sometimes systems need the file to be in a particular line ending so changing that may not be smart.

:e ++ff=mac

You will then be able to see how this file is supposed to look.

For other file formats its similar

:e ++ff=dos
:e ++ff=unix
Timbinous
  • 2,863
  • 1
  • 16
  • 9
1

The command 

sed '/[[:cntrl:]].../s///g ; /[[:cntrl:]]/s///g' typescript > typescript2

works fine for me

Top Maths
  • 61
  • 6
0

^M character comes at the end of line when you write script on windows and port the script on linux machine. To remove this ^M character you can use linux build in command dos2unix as follows:

dos2unix script_name

this will remove all the ^M characters from your script. I have not tested it for ^[ but i am sure that it works for removing ^M.

neo
  • 969
  • 1
  • 11
  • 23
  • have a look here at http://stackoverflow.com/questions/5843495/what-does-m-character-mean-in-vim for other methods to remove these character. – neo Nov 06 '13 at 18:56