How to iterate through a file in Python27 without running into ValueError and completely iterating through file with empty lines?

Question

I essentially have the same problem as this guy : person also having issues iterating

Depending on what I change, I will either run into an IOError, a ValueError (when I use a for each to iterate through each line in the file, and read using readline()), or the program works but it cuts off my data when there's an empty line. I've also tried using the for each loop to iterate through the file with .next() instead of readline, but that skips just about every other line in my data set. I believe top comment there has the solution to my question, except my text file will have lines that are empty, which ends the while loop too early. What is the best way around this? Is there a better data structure to use, or do I have to somehow parse my file to remove empty lines?

Here's a segment of my code, I'm using .rstrip() to get rid of the newline characters at the end of each line:

f = open(self.path,'r')
    while True:
        line = f.readline().rstrip()
        temp_lines_list.append(line)
        if not line:
            break

Some sample input:

text1 : 2380218302
test2 : sad
test3 : moresad (very)
yetanothertest : more datapoints

wowanewsection: incredible

I hope this helps thank you :)

if you say you are having the same issue as another person it is most likely that this is a duplicate of that question. Please explicitly state how this is different. — Ryan Schaefer, Mar 13 '19 at 18:26
If you are using a `for` loop like `for line in f: ...`, there's no need to use `readline` or `next` or any other function to read a line; the iterator already provides each new line via the variable `line`. `next` is implicitly called for you. — chepner, Mar 13 '19 at 18:26
Have you tried using [exception handling](https://docs.python.org/3/tutorial/errors.html#handling-exceptions)? — Irfanuddin, Mar 13 '19 at 18:27
With the `while` loop, you are stripping the new line too soon; the end-of-file condition is marked by `readline` returning a truly empty string, while an empty line from a file comes back as the non-empty string `'\n'`. — chepner, Mar 13 '19 at 18:27

score 1 · Answer 1 · answered Mar 13 '19 at 18:28

1

Have you tried something like this:

lines_output = []
with open('myFile.txt', 'r') as file: # maybe myFile.txt == self.path??
    for line in file.readlines(): # we use readlines() instead of readline() so we iterate entire file
        stripped_line = line.strip()
        if stripped_line not '':
            lines_output.append(stripped_line) # save info if line is not blank
        else:
            pass # if line is blank just skip it

answered Mar 13 '19 at 18:28

Reedinationer

5,661
1
12
33

There's no need to use `readlines`; the file itself is already an iterator. – chepner Mar 13 '19 at 18:29

score 1 · Accepted Answer · answered Mar 13 '19 at 18:29

The readline() method returns a line with a trailing newline character, even on an empty line. You should check if the line is empty before you strip it instead:

while True:
    line = f.readline()
    if not line:
        break
    temp_lines_list.append(line.rstrip())

However, it is more idiomatic in Python to use the file object as an iterable to iterate through the lines of a file, so that you don't have to manage the iterations on your own.

for line in f:
    temp_lines_list.append(line.rstrip())

thank you so much coming from Java I keep forgetting I can iterate through the lines in that way and not have to use readline() or readlines() — Sibel DeShong, Mar 13 '19 at 18:38

How to iterate through a file in Python27 without running into ValueError and completely iterating through file with empty lines?

2 Answers2