Load the first n items from a pickle file

Question

This may be an unusual thing to do, but I would like to load n items from a pckle file, it can be random, or first or last items.

The reason is that I am testing a neural network that load the pickle file as data and I want the network to run a lot faster (reducing from 100000 items to n) in the testing phase.

Right now I am loading the pickle as:

l = pickle.load(open(file, 'rb'))

Is there any parameter to load only the first n items?

The file looks like this:

possibly : https://stackoverflow.com/questions/26860051/how-to-reduce-the-time-taken-to-load-a-pickle-file-in-python — jmunsch, Aug 11 '17 at 04:59

score -5 · Accepted Answer · answered Aug 11 '17 at 05:53

-5

This was quite easy actually:

n = 100;
l = pickle.load(open(file, 'rb'))[1:n];

answered Aug 11 '17 at 05:53

dorien

5,265
10
57
116

4

This still loads the entire file - you're just selecting a portion of them (n-1, incidentally, not n) after you load. – perigon Aug 11 '17 at 05:56

Load the first n items from a pickle file

1 Answers1