How to avoid repeating header when Writing to CSV in loop?

Question

I want to save the values of different variables in a CSV file. But it prints another header every time. I don't want this, I am attaching my CSV file snapshot for your understanding. Output csv

file_orimg = open('Org_image.csv', 'a', newline='')
writer_orimg = csv.writer(file_orimg, delimiter='\t',lineterminator='\n',)
writer_orimg.writerow(["Image Name", "epsilon","MSE", "SSIM", "Prediction", "Probability"])

for i in images:
     writer_orimg.writerow([i, epsilon, mse, ssim, clean_pred, clean_prob, label_idx.item()])

mallocation · Answer 1 · 2020-05-10T09:36:16.400

3

Try not to use writerow to write your headers. You can look at DictWriter in the CSV python module, writing headers and writing rows will be done more efficiently!

list_of_headers = ['No.', 'Image Name', 'Epsilon']
dictionary_content = {'No.': 1, 'Image Name': 'image_123', 'Epsilon': 'what?'}
w = csv.DictWriter(my_csvfile, fieldnames= list_of_headers)
w.writeheader()
w.writerow(dictionay_content)

Hope this helps, let me know if there is any rectification to be made!

Edit: Answering 'where & when should writeheader be done'

I use the os python module to determine whether the file exists, if not I'm going to create one!

if os.path.isfile(filename):
    with open(filename, 'a', newline='') as my_file:
        w = csv.DictWriter(my_file, fieldnames= list_of_headers)
        w.writerow(dictionay_content)
else:
    with open(filename, 'w', newline='') as my_file:
        w = csv.DictWriter(my_file, fieldnames= list_of_headers)
        w.writeheader()
        w.writerow(dictionay_content)

!!! Take note of the 'a' which is to append whereas 'w' means to write. Hence appending with new rows of your data from where it left off/last occupied.

edited May 10 '20 at 09:36

answered May 10 '20 at 07:46

mallocation

526
6
13

it should be in the loop or out of loop? – Faheem-Ur-Rehman 2305-FETBSEEF May 10 '20 at 08:03
Create an if condition to check for the need for writing headers. If headers exist no need to write headers. Empty/new CSV = write headers and you can leave out the writing header from that point on. – mallocation May 10 '20 at 08:29
I've gone ahead and added a section that will answer your loop question. – mallocation May 10 '20 at 08:35
when first iteration is completed and save the value in first row. will it move to next row for next iteration? – Faheem-Ur-Rehman 2305-FETBSEEF May 10 '20 at 08:43
Yes, if the csvfile exists it will append and append and append until you are finished with your images . "for i in images", so in this case first 'i' will write header + contents of 'i', then subsequent 'i' will be on subsequent rows. – mallocation May 10 '20 at 09:02
i am getting (name 'dictionay_content' is not defined) error – Faheem-Ur-Rehman 2305-FETBSEEF May 10 '20 at 09:37
your content should be, dictionary_content = {'Image Name' : i, 'epsilon' : 0.5, 'MSE': 0.051513838, ' SSIM': 0.513531531, 'Prediction': vulture....} Follow this structure > {key: value}, your key would be the headers, your value would be the different variables/values that changes throughout rows! (I'm adding more edits into my answer for ease of readibilityfor future readers) – mallocation May 10 '20 at 09:45
1

i am writing this, because these are not constant values the are variable values which are changing in every iteration, dictionary_content = ({'Image': i, 'epsilon': epsilon ,'MSE': mse, 'SSIM': ssim, 'Prediction': prediction ) – Faheem-Ur-Rehman 2305-FETBSEEF May 10 '20 at 09:49
Hence you got to fill in the variable values into a dictionary before writing a row into the csv. So at first iteration, header row will be written AND the row of variable values will be written. The second iteration, no more writing of headers, just writing variable values, which have changed. First iteration, your image name would be 'abcde', second iteration, your image name would be 'fghij', then next is 'klmno'... So you gotta declare your dict at every iteration to take the new values, and write the current duct. Every iteration, feed an updated dict to writerow! :) – mallocation May 10 '20 at 09:56
what should be the file name in, if os.path.isfile(filename): – Faheem-Ur-Rehman 2305-FETBSEEF May 10 '20 at 10:17
Naming is your personal preference like "image_results.csv". When you run your script and if the filename is not there, it will create one in the same folder the script was running. If you want to choose a specific file to edit and add rows to, remember to add the full file path :) hope this helps Faheem – mallocation May 10 '20 at 10:25
1

Thank you for your time. i got my solution through https://stackoverflow.com/questions/24661525/python-writing-for-loop-result-to-csv – Faheem-Ur-Rehman 2305-FETBSEEF May 11 '20 at 09:10
No problem, all the best! – mallocation May 11 '20 at 09:13

How to avoid repeating header when Writing to CSV in loop?

1 Answers1