16

novice here. I am trying to read lines from a file, however a single line in a .txt file has a \n in the middle somewhere and while trying to read that line with .readline python cuts it in the middle and outputs as two lines.

  • when I copy and past the line to this window, it shows up as two lines. So i uploaded the file here: https://ufile.io/npt3n

  • also added screenshot of the file as it shows in txt file.

  • this is group chat history exported from Whatsup..if you are wondering.
  • Please help me to read one line completely as shown in txt file.

.

f= open("f.txt",mode='r',encoding='utf8')

for i in range(4):
    lineText=f.readline()
    print(lineText)

f.close()

enter image description here

devssh
  • 1,184
  • 12
  • 28
hgv
  • 227
  • 1
  • 2
  • 5
  • 2
    How can a line have `\n` in the middle? `\n` is the thing that separates each line from the next. – khelwood Aug 23 '18 at 07:58
  • 2
    @khelwood not on Windows. It's OS specific. – simon Aug 23 '18 at 08:01
  • Python recognizes `\n` as an eond of line marker. However Windows uses `\r\n`, so a mere `\n` does not split the line in e.g. Notepad. Maybe [this question](https://stackoverflow.com/q/45304492/6337519) might help you with it. – joH1 Aug 23 '18 at 08:05
  • I think its because there is a 'next line character' in the sentence. Maybe, the person on the chat had entered the text 'Kocaeli 24...' in a new line. But the file when on my system shows it on a different line by default on notepad++. So maybe its a issue with notepad. – kkblue Aug 23 '18 at 08:10
  • Python has "universal newlines support". Basically *all* of `\n`, `\r` and `\n\r` are considered a newline. If you open the file in text-mode python will convert those 3 line-endings into just `\n`. If you have to interpret the text differently you want to open the file in binary mode and handle lines by hand. – Giacomo Alzetta Aug 23 '18 at 08:16
  • Related: https://stackoverflow.com/questions/47927039/reading-a-file-until-a-specific-character-in-python – Ciro Santilli OurBigBook.com Sep 27 '21 at 21:42

2 Answers2

18

Python 3 allows you to define what is the newline for a particular file. It is seldom used, because the default universal newlines mode is very tolerant:

When reading input from the stream, if newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller.

So here you should made explicit that only '\r\n' is an end of line:

f= open("f.txt",mode='r',encoding='utf8', newline='\r\n')

# use enumerate to show that second line is read as a whole
for i, line in enumerate(fd):   
    print(i, line)
Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
  • 2
    It can be one of None, '', '\n', '\r', and '\r\n'. I tried giving it "/>\n" for an xml file I had and it gave me a ValueError: illegal newline value. The file is too big to fit in memory by doing a full read, so I can't do that and split. And of the millions of rows I have, one is bound to have a bad "\n" instead of the proper "\\n" as the data has a free text field which is escaped by double quotes. Worst case scenario that line, and a poor neighbour get corrupted as I use regex and drop the line if corrupt. My concern is that poor neighbouring line. – devssh Sep 27 '18 at 07:09
  • 2
    @devssh: It is a different question. I would read lines with standart newline value (`None`) and concat them if last character is not a `"\>"`. But anyway, using regexes for XML is generally a poor solution. BTW, xml.sax can be used to process a xml file without loading everything in memory... – Serge Ballesta Sep 27 '18 at 07:43
  • 1
    Aah, such a good idiom it would have been for storing multiple JSONs in single file for stream-like parsing! Universe is sadistic. – ogurets Oct 24 '19 at 09:58
  • 2
    It should be noted that the only [allowed values](https://docs.python.org/3/library/functions.html#open) for the `newline` argument are `None`, `''`, `'\n'`, `'\r'`, and `'\r\n'`. This may bite you when lines in your file are separated by one of the various [Unicode newline characters](https://en.wikipedia.org/wiki/Newline#Unicode). – Jeyekomon Mar 05 '21 at 13:37
1

Instead of using readline function, you can read whole content and split lines by regex:

import re

with open("txt", "r") as f:
    content = f.read()
    # remove end line characters
    content = content.replace("\n", "")
    # split by lines
    lines = re.compile("(\[[0-9//, :\]]+)").split(content)
    # clean "" elements
    lines = [x for x in lines if x != ""]
# join by pairs
lines = [i + j for i, j in zip(lines[::2], lines[1::2])]

If all content has the same beginning [...] you can split by this, then clean all parts omitting the "" elements. Then you can join each part with zip function (https://stackoverflow.com/a/5851033/1038301)

Roomm
  • 905
  • 11
  • 23