How do I continue on error in loop condition in python?

Question

I have python code that loops over a file. I'm getting a UTF-8 error (invalid continuation byte) when I read over the file. I just want my program to ignore that.

I've tried using a try except around the code inside, but that won't work since the error is in the condition of the for loop. I've also tried using a try except around the loop but then when it catches the error it doesn't start the loop again.

with open(input_file_path, "r") as input_file:
    for line in input_file:
        # code irrelevant to question

What happens is it gives this error on for line in input_file:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 5: invalid continuation byte`

I want it to skip that line and move to the next one. Essentially, a try catch on the condition of my for loop.

Are you sure the file *should* be decoded as a UTF-8 stream? Adding the correct encoding the `open` function may be the correct answer. — chepner, Oct 05 '19 at 16:20

chk · Accepted Answer · 2019-10-05T16:15:00.310

3

Does this work? (edited to solution OP found)

with open(input_file_path, "r", encoding="utf8", errors="surrogateescape") as input_file:
    for line in input_file:
        try:
            yourcode
        except:
            continue

edited Oct 05 '19 at 16:15

answered Oct 05 '19 at 15:44

chk

58
4

Yes I did and it doesn't work since te error is on the line `line in input_file` – Sheshank S. Oct 05 '19 at 15:45
then its not a line issue but prob a file issue? is the encoding correct? – chk Oct 05 '19 at 15:51
a quick search came up with this. https://stackoverflow.com/questions/19699367/unicodedecodeerror-utf-8-codec-cant-decode-byte – chk Oct 05 '19 at 15:52
Yes I did see that. – Sheshank S. Oct 05 '19 at 16:02
If you change your answer to this: `with open(input_file_path, "r", encoding="utf8", errors="surrogateescape") as input_file:` I'll accept it. That worked for me – Sheshank S. Oct 05 '19 at 16:02

BramAppel · Answer 2 · 2019-10-05T15:50:45.400

1

Have you tried something like this, when the UnicodeDecoceError is raised, the loop will continue with the next iteration.

with open(input_file_path, "rb") as input_file:
    for line in input_file:
        try:
            line_i = line.decode(encoding='utf-8')
        except UnicodeDecodeError:
            continue

edited Oct 05 '19 at 15:50

answered Oct 05 '19 at 15:44

BramAppel

1,346
1
9
21

I tried that, but it doesn't work since the error is on the line `for line in input_file` – Sheshank S. Oct 05 '19 at 15:46
@SheshankS. Allright, maybe then read the file content as binary and convert at a later moment. – BramAppel Oct 05 '19 at 15:49

score 0 · Answer 3 · answered Oct 05 '19 at 16:52

0

You can use

with open(input_file_path, "r", encoding="ISO-8859-1") as input_file:
    for line in input_file:

answered Oct 05 '19 at 16:52

Sudhansu Kumar

129
1
3
10

How do I continue on error in loop condition in python?

3 Answers3