What is the best way to loop through a huge csv file

Asked Mar 07 '16 at 09:11

Active Nov 12 '18 at 04:55

Viewed 512 times

I am making a system that is reading huge csv files and is storing the data of them in a database. With huge files I mean 90 csv files with 30000 rows in each file. And every row has around the 30-40 columns.

I was wondering what the fastest way was to loop trough all those files.

Currently I was using the default build-in csv reader. But I dont know if that is the fastest/most efficient way.

Does someone know a better way?

edited Nov 12 '18 at 04:55

Cœur

37,241
25
195
267

asked Mar 07 '16 at 09:11

user3824329

2

http://stackoverflow.com/questions/11077801/import-csv-to-mysql-table might help – Alex Andrei Mar 07 '16 at 09:13
1

I had the same task a few weeks ago and what I did was use multi-threading. I don't know if it's the best approach but it does my task faster. – Lex Bryan Mar 07 '16 at 09:15
1

Check out this answer — http://stackoverflow.com/a/13287221/487923 – ksimka Mar 07 '16 at 09:25

What is the best way to loop through a huge csv file

0 Answers0