0
  1. read as follows
c_size=10000000
c_chunk = pd.read_csv("CHARTEVENTS.csv", index_col=0, chunksize=c_size)
  1. wanted here each chunk in df format
c_list= []
for chunk in c_chunk:
    chunk.columns=['ROW_ID', 'SUBJECT_ID', 'HADM_ID', 'ICUSTAY_ID', 'ITEMID', 'VALUE', 'VALUENUM', 'VALUEUOM']
    c_list.append(chunk)
  1. the plan is to append the chunk to list. however i get an empty list, what do you think happened 
c_chunk = pd.concat(c_list)
  1. As a result of empty list, i am getting valueError: no objects to concatenate. Your help is appreciated.
Abe
  • 1
  • 2
  • 1
    `chunk_list` vs `c_list` typo? – tevemadar Nov 16 '19 at 13:35
  • sorry that is typo when i was posting the question, not in the main code. Thank you. I will edit. – Abe Nov 16 '19 at 13:52
  • Can you post a small (10 rows by 5 columns) sample of your CSV file? This seems at first glance like it should work... – NicholasM Nov 16 '19 at 14:02
  • I tried your code with my own csv file, and it works fine. Check this question, maybe it helps you. https://stackoverflow.com/questions/25962114/how-to-read-a-6-gb-csv-file-with-pandas – Amir Nov 16 '19 at 15:20
  • is your `pd.read_csv()` within the for loop? also are you renaming the columns with the `.columns` argument? – Umar.H Nov 16 '19 at 15:41

0 Answers0