Python how to convert a newline character to int()

Question

I have several lists that I want to save in a .txt file, all on a new line. So, I have

list1 = [12, 32, 233]
list2 = [345, 823, 209]

and so on... So I do

with open("textfile.txt", "w") as f:
    f.write(str(list1))
    f.write("\n")
    f.write(str(list2))

But when I try to read the file, as int(), I get an error saying: invalid literal for int() with base 10 because the program thinks the \n is a character too and cannot convert it to int() How can I solve this issue? Reading line:

with open("textfile.txt", "r") as f:
    data = f.readlines()
    actualList1 = int(data[0])
    actualList2 = int(data[1])

Please include the code that you are using to read the file. It should include the line creating issue and other relevant lines — Melvin Abraham, Nov 09 '20 at 18:23
Does this answer your question? [How to read a file without newlines?](https://stackoverflow.com/questions/12330522/how-to-read-a-file-without-newlines) — Random Davis, Nov 09 '20 at 18:23
he doesnt need to reproduce a sample of the text file since he shows the code that generates it — Ironkey, Nov 09 '20 at 18:31
Storing the representation of a list isn't the best of ideas if you want to read it back. — Matthias, Nov 09 '20 at 19:08

Ironkey · Accepted Answer · 2020-11-09T19:02:45.007

2

you can take it apart and store it delimited with spaces and then put it back together when reading it like so:

list1 = [12, 32, 233]
list2 = [345, 823, 209]

with open("textfile.txt", "w") as f:
    for lst in [list1,list2]:
        f.write(" ".join([str(i) for i in lst]) + "\n")

with open("textfile.txt", "r") as data:
    data = [i.strip() for i in data.readlines()]
    x = [[int(num) for num in i.split()] for i in data]

print(x)

[[12, 32, 233], [345, 823, 209]]

So what's going on?

well let's break it down into 2 steps; storing the values and retrieving the values

part 1

with open("textfile.txt", "w") as f:
    for lst in [list1,list2]:
        f.write(" ".join([str(i) for i in lst]) + "\n")

all this does is store the list separated by spaces

Using the join function we can easily accomplish this with one workaround; since the values stored inside the list are integers they must be strings when used with this function. Hence the use of the following list comprehension: [str(i) for i in lst]!

With that string, we can finally add a newline character to make sure we don't have one line of everything combined!

part 2

with open("textfile.txt", "r") as data:
    data = [i.strip() for i in data.readlines()]
    x = [[int(num) for num in i.split()] for i in data]

This is a little bit more complicated, in essence, this creates a list of the lines of the text file, and then strips them of newline characters.

From this, we split each line by spaces, and then cast each string to an integer value inside our newly created list.

edited Nov 09 '20 at 19:02

answered Nov 09 '20 at 18:24

Ironkey

2,568
1
8
30

2

It's better to avoid the use of `eval` due to its unsafe nature – Melvin Abraham Nov 09 '20 at 18:26
its only unsafe if he is letting other users input to it, not if he's storing the code himself... – Ironkey Nov 09 '20 at 18:26
1

@Ironkey, funny thing, the worst data loss event I've been present for was a case where folks used that same reasoning to justify sloppy practices (and then a buffer overflow in a program writing data files caused garbage to be injected into filenames, which are what were being unsafely processed). – Charles Duffy Nov 09 '20 at 18:32
Anyhow, why would you use `eval` when you could instead use [`ast.literal_eval`](https://docs.python.org/3/library/ast.html#ast.literal_eval) and get the same ability to parse Python literal data, but _not_ need to worry about there being code mixed into it? – Charles Duffy Nov 09 '20 at 18:39
Thanks for the eval answer! And as.literal_eval. But what does .strip() do? – LMCuber Nov 09 '20 at 18:41
`strip` just removes newline characters among other things, that way its not trying to interpret `"223\n"` as an int! – Ironkey Nov 09 '20 at 18:42
3

Why don't you just use [json](https://docs.python.org/3/library/json.html#module-json)? Trying to parse python literals from files is a mug's game... – ekhumoro Nov 09 '20 at 18:44
1

though text file parsing is still some good practice for beginners, he's right you could easily just store your lists there using methods like `json.dump` – Ironkey Nov 09 '20 at 18:45
3

@Ironkey Text file parsing is a good skill to learn, but learing which tool is right for the job will be much more valuable in the long run. This is especially true when it comes to `eval`, which is hardly ever the right tool for anything, and carries serious risks for newbies. Please don't ever suggest it when there are better and safer alternatives available. – ekhumoro Nov 09 '20 at 18:53
1

edited to match then, ill add an explanation shortly – Ironkey Nov 09 '20 at 18:56

Yovel · Answer 2 · 2020-11-09T18:48:26.617

list1 = [12, 32, 233]
list2 = [345, 823, 2209]

with open("textfile.txt", "w") as f:
    f.write(str(list1))
    f.write("\n")
    f.write(str(list2))

with open("textfile.txt", "r") as f:
    data = f.read()# since every line but the last ends with a '\n' I rather read than split to lines 
data = data.splitlines()
#you list is a string now so I split it like one
actualList1 = [int(x) for x in data[0][1:-1].split(',')]# split from 1 to -1 will remove parenthesize, after I split by ',' because its seperates the values
actualList2 = [int(x) for x in data[1][1:-1].split(',')]

print(actualList1)
print(actualList2)

Output:

[12, 32, 233]
[345, 823, 2209]

score 0 · Answer 3 · answered Nov 09 '20 at 18:52

Please note how the conversion issue is due also to '[' and ']' characters.

What you need is to strip all these characters (including also '\n') and then to split these numeric strings into a list. Finally you just have to convert them to int.

In this example we know that the file has at least two rows (this code is similar to yours):

list1 = [12, 32, 233]
list2 = [345, 823, 209]

with open("textfile.txt", "w") as f:
    f.write(str(list1))
    f.write("\n")
    f.write(str(list2))


with open("textfile.txt", "r") as f:
    data = f.readlines()
    actualList1 = [int(v) for v in data[0].strip("[]\n").split(",")]
    actualList2 = [int(v) for v in data[1].strip("[]\n").split(",")]
    print(actualList1)
    print(actualList2)

Output:

[12, 32, 233]
[345, 823, 209]

The previous snipped applied the fix to your code, but take a look to this solution, which works also with a variable number of rows (besides it is more elegant):

with open("textfile.txt", "r") as f:
    data = f.readlines()
    for d in data:
        actualListRow = [int(v) for v in d.strip("[]\n").split(",")]
        print(actualListRow)

Python how to convert a newline character to int()

3 Answers3