-1

When I want to call pandas.read..function, what is the most memory efficient reader/writer, which would save the most memory on my machine? I want to read a large datafile and wonder if I can pick a format upfront to save the data with some other tool to save some memory consumption in python.

Is it: pd.read_csv, pd.read_hdf, ..?

The fast file is explained here: Fastest file format for read/write operations with Pandas and/or Numpy

is this also the most memory efficient way?

PV8
  • 5,799
  • 7
  • 43
  • 87

1 Answers1

1

Why don't you use modin library. It is meant to speed up most of the data frame operations. You can find more information here at this link https://github.com/modin-project/modin.

ashish14
  • 650
  • 1
  • 8
  • 20