How to remove ^M

Question

How can I remove the ^M character from a text file (at the end of line) in a Python script?

I did the following, and there are ^M at every line-break.

file = open(filename, "w")
file.write(something)

Can you post more off your code?. Dont`t know where ^M comes from. New line character make and "\n" not "^M" Remeber to close "file.close()" Dont use file is an reserved word in python use my_file or something. — snippsat, Jul 07 '10 at 02:04
@snippsat ^M is the terminal character escape which is written as '\n' in C syntax — Pete Kirkham, Jul 07 '10 at 09:16
@Ned you're right - I was thinking of the old days where if you typed Ctrl+M you get a new line, but that's because it's carriage return not newline — Pete Kirkham, Jul 07 '10 at 13:22

Chris B. · Accepted Answer · 2010-07-07T02:02:35.260

9

If you're writing the file, you should specify open(filename, "wb"). That way, you'll be writing in binary mode, and Python won't attempt to determine the correct newlines for the system you're on.

edited Jul 07 '10 at 02:02

answered Jul 07 '10 at 01:35

Chris B.

85,731
25
98
139

Thanks for your reply, but I'm trying to write 'something' to 'file' and it seems 'Universal newline mode' can only be used with 'rU', if I'm not mistaken. It doesn't work for me, anyways. – DGT Jul 07 '10 at 01:55

score 6 · Answer 2 · answered Feb 28 '13 at 14:33

6

string.replace('\r', '') worked for me.

Ugly, but nor r+ nor r+b nor NOTHING ELSE worked (for me, sure) :(

answered Feb 28 '13 at 14:33

inigoD

1,681
14
26

This was the only solution that worked for me. I eventually figured out why. The '\r' was not being introduced when writing the file. Instead, they were in the original file being read in. Thus this removed it, but none of the other solutions would. – mikemtnbikes Sep 20 '22 at 01:38

score 6 · Answer 3 · answered Jul 07 '10 at 02:12

6

Python can open a file in binary mode or in text mode. Text is the default, so a mode of "w" means write in text mode. In text mode, Python will adjust the line endings for the platform you're on. This means on Windows, this code:

f = open("foo.txt", "w")
f.write("Hello\n")

will result in a text file containing "Hello\r\n".

You can open the file in binary mode by using "b" in the mode:

f = open("foo.txt", "wb")
f.write("Hello\n")

results in a text file containing "Hello\n".

answered Jul 07 '10 at 02:12

Ned Batchelder

364,293
75
561
662

Actually, that 'something' comes from html form textarea, where I copy and paste 'something'. The script then gets the value: something = form["some_name"].value – DGT Jul 07 '10 at 02:59

score 4 · Answer 4 · answered Oct 25 '21 at 09:54

How to explicitly set carriage return when doing json.dump? contains the solution: the open function has an extra parameter newline:

file = open(filename, "w", newline="\n")
file.write(something)

newline controls how universal newlines mode works (it only applies to text mode). It can be None, '', '\n', '\r', and '\r\n'. It works as follows:

When reading input from the stream, if newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller. If it is '', universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.

When writing output to the stream, if newline is None, any '\n' characters written are translated to the system default line separator, os.linesep. If newline is '' or '\n', no translation takes place. If newline is any of the other legal values, any '\n' characters written are translated to the given string.

score 2 · Answer 5 · answered Jul 05 '15 at 02:58

2

For portability, you can try the following

import os
file = open(filename, "w")
file.write(something.replace('\r\n', os.linesep))

answered Jul 05 '15 at 02:58

knight42

910
10
15

score 2 · Answer 6 · answered Jul 09 '10 at 10:18

2

dos2unix filename.py

to convert the line breaks to UNIX style.

answered Jul 09 '10 at 10:18

bigredbob

1,847
4
19
19

score -1 · Answer 7 · answered May 28 '16 at 15:50

-1

run autopep8 on the file

> apt-get install python-autopep8

> autopep8 python_file_name > new_python_file_name.py

answered May 28 '16 at 15:50

madscatt

34
6

This is not what he's asking. This would also probably have other side effects; answer the question asked with detail and real solutions. – kevr Jun 23 '17 at 20:21

score -1 · Answer 8 · answered May 09 '23 at 11:22

-1

To fix (normalize) all files in a repository you can also run

git add --renormalize .

this worked best for without having to write a script to fix the files. To make sure line endings show in git status you can set this:

git config --global core.autocrlf input

on a Mac book.

answered May 09 '23 at 11:22

puclanac

114
1
4

This question is not about git. – Friedrich May 09 '23 at 11:47
@Friedrich That's true, but it's still a way to solve the question. I had the same problem and needed to fix multiple files, it was easier to run this git command instead of writing a python script that loops over the files and fixes them. – puclanac May 10 '23 at 14:25
I'm glad it worked for you. I don't think it's generally a good solution for the following reasons: (1) it will only work in a git repo. (2) it will not work on the *output of a script* unless you `git add` it. (3) it will not prevent CRLFs from being written, only fix them in another pass. (4) it depends on global git config (yeah sure, it needn't). (5) it does absolutely nothing that `dos2unix` wouldn't do only with much more effort. – Friedrich May 10 '23 at 14:40

How to remove ^M

8 Answers8

Linked