Saving Multiple files in Python

Question

I am trying to create a new file each time the following runs. At the moment it creates 1 file and just overwrites it. Is there a to make it not overwrite and create a new file for each loop?

import xml.etree.ElementTree as ET
import time
import csv

with open('OrderCSV.csv', newline='') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        orders_data = ET.Element('orders_data')
        orders = ET.SubElement(orders_data, 'orders')

        ##Order Details
        order_reference = ET.SubElement(orders, 'order reference')
        order_reference.set('',"12345")
        order_date = ET.SubElement(order_reference, 'order_date')
        order_priority  = ET.SubElement(order_reference, 'order_priority')
        order_category = ET.SubElement(order_reference, 'order_category')
        delivery_service = ET.SubElement(order_reference, 'delivery_service')
        delivery_service.text = row['delivery_service']

        timestr = time.strftime("%Y%m%d%H%M%S")

        mydata = ET.tostring(orders_data)
        myfile = open(timestr, "wb")
        myfile.write(mydata)

I fixed some indentation errors... did I get it right? Is the file write in the for loop? — tdelaney, Oct 27 '20 at 22:45
The question was whether I got the indentation right (the file writes are in the for loop) and how fast they are processed. I think you generate multiple a second so your calculated timestamp stays the same. I also assume you had more than one to process. — tdelaney, Oct 27 '20 at 23:07
@Mayamiko You need to [format the time with milliseconds](https://stackoverflow.com/q/7588511/984421) to get unique filenames. — ekhumoro, Oct 27 '20 at 23:11

score 2 · Accepted Answer · answered Oct 27 '20 at 23:05

You could see if the file already exists and wait a bit

    while True:
        timestr = time.strftime("%Y%m%d%H%M%S")
        if not os.path.exists(timestr):
            break
        time.sleep(.1)
    with open(timestr, "wb") as myfile:
        mydata = ET.tostring(orders_data)
        myfile.write(mydata)

Instead of waiting you could just add seconds. This will cause the file names to drift forward in time if you process a lot of them per second.

    mytime = time.time()
    while True:
        timestr = time.strftime("%Y%m%d%H%M%S", time.localtime(mytime))
        if not os.path.exists(timestr):
            break
        time.sleep(.1)
    with open(timestr, "wb") as myfile:
        mydata = ET.tostring(orders_data)
        myfile.write(mydata)

Another option is to get a single timestamp before the loop and update it as you go.

mytime = time.strftime("%Y%m%d%H%M%S")
for index, row in enumerate(reader):
     ....
     mytime = f"mytime-{index}"
     ....

Okay, why did that end up as a different answer?! Pilot error I suppose. — tdelaney, Oct 27 '20 at 23:16

score 0 · Answer 2 · answered Oct 27 '20 at 22:40

0

change the variable name each time you run the loop and I would suggest using with statement for opening file as you also have to close it after you open it

with open(timestr, 'wb') as myfile:
    myfile.write(mydata)

edit: only flaw I can imagine in your code is not closing the file after opening it

answered Oct 27 '20 at 22:40

Matiiss

5,970
2
12
29

I believe the timestr creates a unique filename each time but it is just overwriting the old one – Mayamiko Oct 27 '20 at 23:04
@Mayamiko try different naming (convention?) like str(row) – Matiiss Oct 27 '20 at 23:06
@Mayamiko have you tried using with statement? i fthat does not work use something simple like c = 0 and filename is str(c) and add up the c – Matiiss Oct 27 '20 at 23:09

Saving Multiple files in Python

2 Answers2