2

I have a simple Python script that writes a sub(CSV)file after reading a large CSV file.

import csv
import os

large_file = "D:\Total_data.csv"

with open(large_file, 'rb') as csvfile:

     reader = csv.reader(csvfile, delimiter='|')

     path = "D:/CSV_Files/"
     if not os.path.exists(path):
        os.makedirs(path)

     writer1 = csv.writer(open(path+"A1_subfile"+".csv", 'wb'), delimiter = '|')

     #writing data to the CSV file only if first column has value 'A1'
     for row in reader:
         if (row[0]=='A1'):
             writer1.writerow(row)
csvfile.close()

The problem I am facing is that when I run the script for first time, some rows don't come in A1_subfile.csv file(say only 96 or 97 rows are shown out of 100 rows).

When I run the exact SAME script for the second time, then all rows are shown in the subfile(now all 100 rows can be seen).

Even if I wait for some minutes after executing for the first time, I don't get complete output in new CSV file. And if I simply execute code 2 times, I can see complete output immediately.

I have searched, but couldn't get over this issue. Please help to overcome this.

Aditya
  • 615
  • 3
  • 12
  • 26
  • Why are you opening one file using `with` but not the other? Make sure to close your file properly. – Aran-Fey Aug 22 '16 at 07:46
  • 1
    Possible duplicate of [CSVWriter not saving data to file - WHY?](http://stackoverflow.com/questions/3976711/csvwriter-not-saving-data-to-file-why) – cdarke Aug 22 '16 at 08:00
  • 1
    Also http://stackoverflow.com/questions/13675072/csvs-writerow-in-python-doesnt-work-most-of-the-time and http://stackoverflow.com/questions/11454453/writing-csv-files-in-python-files-are-left-blank-at-the-end-of-the-operation – cdarke Aug 22 '16 at 08:03
  • Sorry but the problem continues even after using flush() and close() methods, as described in other answers – Aditya Aug 22 '16 at 08:18

1 Answers1

-1

I was randomly trying many different ways to solve this issue, and interestingly I found that if we define a function for this code, and then call the function, the issue gets resolved.

def create_subfile():

     """ now same code as in question"""

create_subfile();

Does anyone know the reason why a piece of code works fine when executed through a function call, and doesn't work when executed directly ?

Aditya
  • 615
  • 3
  • 12
  • 26
  • seriously, close the output file or using context manager for the output file will solve the question. – xmcp Aug 22 '16 at 09:48
  • I closed the output file too. But the code worked only when it was written inside the function. Don't know reason for such "strange" solution – Aditya Aug 22 '16 at 09:51
  • May I know who has downvoted this answer, and why? I know it's a strange solution, but it is a solution to the problem. – Aditya Sep 09 '16 at 17:56