invalid literal for int() with base 10: '' while parsing text

Question

I am processing a text file with 6400 numbers separated by a comma in an 80 x 80 matrix, however I am getting an;

invalid literal for int() with base 10: ''

What's strange is that this code works on windows but not mac (same python version)

I've tried checking for extra spaces and commas by using try and except to no avail

def read_numbers(file):
    with open(file) as f:
        text = f.read()
        text = text.replace("\n", ",")

    numbers = []
    for s in text.split(','):
        try:
            numbers.append(int(s))
        except ValueError, e:
            print "error",e, "with", "(",s,")"
    return numbers

Nevertheless, you clearly have a line which contains an empty space between two commas or alone on a line. Looks like at the very least the final newline would be problematic. — tripleee, Feb 13 '19 at 08:00
Have you checked what line separators are used in your file? Is it really a single `\n` or is there some `\r` somwhere? — piet.t, Feb 13 '19 at 08:07

DirtyBit · Answer 1 · 2019-02-13T08:25:32.507

This text = text.replace("\n", ",") is not needed, neither should you do for s in text.split(',')::

numbers = []
def read_numbers(file):
    with open(file) as f:
        content = f.read()

        # you may also want to remove empty lines
        content = [l.strip() for l in content if l.strip()]

    for s in content:

        try:
            numbers.append(int(s))
        except ValueError:
            print("error with {}".format(s))

    return numbers

EDIT:

I tried to reproduce your problem with the following:

list.txt:

12,43,54,65,7676,87,9898,0909,676,46556

2342,6556,7687
5465,76878,98090,9090,5656


33,434,3435,4545 ,5656

and then:

numbers = []
def read_numbers(file):
    with open(file, 'r') as f:
        content = f.read()
    for s in content.split(","):

        if ',' in s or '\n' in s:
            continue
        else:
            numbers.append(s)

    return numbers


print(read_numbers("list.txt"))

OUTPUT:

['12', '43', '54', '65', '7676', '87', '9898', '0909', '676', '6556', '76878', '98090', '9090', '434', '3435', '4545 ', '5656']

score 1 · Answer 2 · answered Feb 13 '19 at 08:05

According to this answer, your source code only supports Window version. So I guess that your s variable at this line numbers.append(int(s)) include newline character on its.

Document here: The key point from that documentation is this

Newline controls how universal newlines works (it only applies to text mode). It can be None, '', '\n', '\r', and '\r\n'. It works as follows:

On input, if the newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller. If it is '', universal newline mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.

On output, if the newline is None, any '\n' characters written are translated to the system default line separator, os.linesep. If newline is '', no translation takes place. If newline is any of the other legal values, any '\n' characters written are translated to the given string.

Solution:

# When you open a file in text mode in Python 3, It will convert all newlines to '\n' and be done with it.
def read_numbers(file):
    with open(file, 'r') as f:
        text = f.read()
        text = text.replace("\n", ",")

    numbers = []
    for s in text.split(','):
        try:
            numbers.append(int(s))
        except ValueError, e:
            print "error",e, "with", "(",s,")"
    return numbers

score 1 · Accepted Answer · answered Feb 13 '19 at 09:41

You might have a trailing , on the end of a line. When the newline is converted into a , this would result in ,,. Your split(',') would then result in a list containing an empty string. This would cause the invalid literal exception.

As an alternative, you could use a regular expression to extract all the integers. This would then automatically cope with possible blank lines or missing values:

import re

def read_numbers(filename):        
    with open(filename) as f:
        return list(map(int, re.findall(r'\d+', f.read())))

@po-chen-liu Why did you have a trailing `,` at the end of a line, `this code works on windows but not mac`? — Trần Đức Tâm, Feb 14 '19 at 02:25

invalid literal for int() with base 10: '' while parsing text

3 Answers3