Manipulating data of file several times

Question

I'm having a little bit trouble with file operations.. I want to manipulate data 2 times and write the last manipulated data into same file.

This is my manipulated file:

['Simon', 93.45000000000002, 'AA']
['Jane', 87.55000000000001, 'BA']
['Peter', 85.95, 'BA']

And here is my code for how to change data in file.

for line in manipulated_file:
    line=line.split()
    list_to_be_written=[]
    for i in line:
        i=i.replace("['","")
        i=i.replace("'","")
        i=i.replace(",",":")
        i=i.replace("]","")
        list_to_be_written.append(i)
    list_to_be_written=" ".join(list_to_be_written)
    print list_to_be_written

It runs perfecly how ever i can't write the output to the file..

Expected output in the same file:

Simon: 93.45000000000002: AA
Jane: 87.55000000000001: BA
Peter: 85.95: BA

Thank you in advance.

Ahsanul Haque · Answer 1 · 2016-01-02T18:41:49.910

3

You can use ast.literal_eval

from ast import literal_eval

with open('demo.txt','r') as input_file, open('output.txt','a') as output:
    for line in input_file:
        temp = literal_eval(line)
        output.write(": ".join([repr(a) if type(a)== float else a for a in temp])+'\n')

Content of output.txt:

Simon: 93.45000000000002: AA
Jane: 87.55000000000001: BA
Peter: 85.95: BA

edited Jan 02 '16 at 18:41

answered Jan 02 '16 at 18:23

Ahsanul Haque

10,676
4
41
57

Padraic Cunningham · Answer 2 · 2016-01-02T19:13:10.010

2

You can use ast.literal_eval to parse the lists and fileinput.input to change the original file.

import fileinput
from ast import literal_eval
import sys


for line in fileinput.input("in.txt", inplace=1):
    sys.stdout.write(": ".join(map(str, literal_eval(line))) + "\n")

Which will give you:

Simon: 93.45000000000002: AA
Jane: 87.55000000000001: BA
Peter: 85.95: BA

You can also do it by just splitting and stripping:

import fileinput
import sys
for line in fileinput.input("in.txt", inplace=True):
    sys.stdout.write(":".join([w.strip("'") for w in line.strip("[]\n").split(", ")]) + "\n"

Which gives you the same output:

Simon: 93.45000000000002: AA
Jane: 87.55000000000001: BA
Peter: 85.95: BA

If the code happened to error you would lose data so a better option may be to write to a tempfile and then do a shutil.move:

from shutil import move
from tempfile import NamedTemporaryFile

with open("in.txt") as f, NamedTemporaryFile("w", dir=".", delete=False) as tmp:
    for line in f:
        tmp.write(": ".join([w.strip("'") for w in line.strip("[]\n").split(", ")]) + "\n")

move(tmp.name,"in.txt")

edited Jan 02 '16 at 19:13

answered Jan 02 '16 at 18:44

Padraic Cunningham

176,452
29
245
321

@PadraicCunnigham .. a silly question, is `stdout` always directed to a file?..how do you make sure of that? – Iron Fist Jan 02 '16 at 18:47
I think you will lose precision in float to str conversion. – Ahsanul Haque Jan 02 '16 at 18:51
@AhsanulHaque, If you look at the OPs values I don't think that is something they are worrying about. – Padraic Cunningham Jan 02 '16 at 18:55
1

@IronFist, fileinput hogs stdout so every print or stdout write gets directed to the file – Padraic Cunningham Jan 02 '16 at 18:59
@PadraicCunningham, I don't know if the precision is important or not, but I commented just because you claimed the code will produce the below output, which is wrong and can create confusion. – Ahsanul Haque Jan 02 '16 at 19:00
1

@AhsanulHaque, that is exactly what it produces in python3 so there is no confusion. You can format it to whatever precision you want in pytonn2 – Padraic Cunningham Jan 02 '16 at 19:00
Haha thanx guys for all your help and comments. I need study hard to code like this:3. Using replace is not efficent when you don't want to change all "," 's or else.. is there any way to use replace method just for a specific character which appears many times in line, .. @PadraicCunningham – B.Joelene Jan 03 '16 at 20:45
@B.Joelene, really how you store you data is key, if you store it in a manner that is easy to parse like a comma separated csv life will be much easier, how did you get the file into the format it is? – Padraic Cunningham Jan 03 '16 at 21:05

score 1 · Accepted Answer · edited May 23 '17 at 12:04

1

I believe that if you want your data will be written into the same file as the initial data, you have to open it first for reading, read into variable, process it, then open it for writing and write down everything that is needed. See the docs for details.

with open("file.txt", "r") as manipulated_file:
    manipulated_file_lines = manipulated_file.readlines()
with open("file.txt", "w") as  manipulated_file:

    for line in manipulated_file_lines:
        line=line.split()
        list_to_be_written=[]
        for i in line:
            i=i.replace("['","")
            i=i.replace("'","")
            i=i.replace(",",":")
            i=i.replace("]","")
            list_to_be_written.append(i)
        list_to_be_written=" ".join(list_to_be_written)
        print list_to_be_written

        manipulated_file.write(list_to_be_written)

P.S. I'm not sure that you have to process your lines manually in this way. I believe it is better to convert them to Python object (e.g. list) with json decoder, then format output string from that object. This can be used instead of those .replace()'s:

import json
with open("file.txt", "r") as manipulated_file:
    manipulated_file_lines = manipulated_file.readlines()
with open("file.txt", "w") as  manipulated_file:
    for line in manipulated_file_lines:
        processed_list = json.loads(line.replace("'",'"'))
        list_to_be_written=": ".join(map(str,processed_list))
        # this should be tweaked if you want full precision for numbers

        print list_to_be_written

        manipulated_file.write(list_to_be_written)

You can also use ast.literal_eval instead of json.loads, see the docs and this question.

edited May 23 '17 at 12:04

Community

1
1

answered Jan 02 '16 at 18:26

Ilya V. Schurov

7,687
2
40
78

Thanks Ilya that is really helpful for me! :) I'm so new in python so that i don't know a lot about decoder or some other methods .. However, i will look at it. Thanks :3 – B.Joelene Jan 02 '16 at 18:48
This won't produce expected result, it will lose precision. – Ahsanul Haque Jan 02 '16 at 18:48
@AhsanulHaque, I agree, but this can be simply fixed by using appropriate `.format()` method or `%` operator, if it is really needed. – Ilya V. Schurov Jan 02 '16 at 18:51
@IlyaV.Schurov by the way im wondering if is it possible after manipulating data 2 times, changing a unicode in file? – B.Joelene Jan 03 '16 at 20:17
For example i changed "," 's with ":" but what about changing one of them " ; " and one of them" : " is it possible @IlyaV.Schurov – B.Joelene Jan 03 '16 at 20:20
@B.Joelene, I'm not sure I understand what do you want. Could you please post the initial file and the desires result? – Ilya V. Schurov Jan 03 '16 at 22:51
yeah sure for example: Simon: 93.45000000000002: AA Jane: 87.55000000000001: BA Peter: 85.95: BA to Simon: 93.45000000000002; AA Jane: 87.55000000000001; BA Peter: 85.95; BA in my code i replace all of the commas with ":" is there a way to do this.. @IlyaV.Schurov – B.Joelene Jan 04 '16 at 16:29

Manipulating data of file several times

3 Answers3