I'm trying load huge dataset into sklearn system via a txt file. Or is there a easy method to upload it
Asked
Active
Viewed 29 times
-1
-
This is far too broad/vague. – AMC Jan 22 '20 at 00:29
1 Answers
0
What's the problem do you have to read it? As a general suggestion you can read it with chunks:
chunksize = 10000
for chunk in pd.read_csv(filename, chunksize=chunksize):
# do whatever you suppose to
Specify the chunksize parameter properly to your PC configuration. You may read this question for more options.

Charnel
- 4,222
- 2
- 16
- 28