-1

I'm trying load huge dataset into sklearn system via a txt file. Or is there a easy method to upload it

1 Answers1

0

What's the problem do you have to read it? As a general suggestion you can read it with chunks:

chunksize = 10000
for chunk in pd.read_csv(filename, chunksize=chunksize):
    # do whatever you suppose to

Specify the chunksize parameter properly to your PC configuration. You may read this question for more options.

Charnel
  • 4,222
  • 2
  • 16
  • 28