1

I have this error: EOFError: Ran out of input when unpicking multiply objects in Python. I basically want to save my 4 randomly generated objects' data in a text file in order to use them later without randomly generating the objects again. There are four objects I am pickling into "data.txt".

import pickle
with open("data.txt", "wb") as output:
    pickler = pickle.Pickler(output)
    pickler.dump(x_data)
    pickle.dump(err, output, protocol=pickle.HIGHEST_PROTOCOL)
    torch.save(err, 'data.txt')
    pickle.dump(x_testdata, output, protocol=pickle.HIGHEST_PROTOCOL)
    torch.save(x_testdata, 'data.txt')
    pickle.dump(y_testdata, output, protocol=pickle.HIGHEST_PROTOCOL)
    torch.save(y_testdata, 'data.txt')
    # pickler.dump(x_testdata)
    # pickler.dump(y_testdata)

output.close()

But when I ran the code:

with open("data.txt", "rb") as input:
    print(pickle.load(input))  # printing first object in this file
    print(pickle.loads(input.read()))  # printing 2nd object in this file
    print(pickle.loads(input.read()))  # printing 3rd object in this file
    print(pickle.loads(input.read()))  # printing 4th object in this file

input.close()

There is always this error: EOFError: Ran out of input when loading the 3rd object in this file. I don't really understand why there's this error since I pickled 4 objects into this file.

Console Output of Error

Here's my entire code:

import numpy as np
import torch
# from sklearn.model_selection import KFold
# from sklearn.svm import SVR
import pickle

n = 50  #number of input(x) observations
p = 2  #dimension of each input(x) observations

x_data = np.random.uniform(0.0, 1.0, n*p)
x_data = np.reshape(x_data, (n, p))
# n observations of p dimension
x_data = torch.from_numpy(x_data)

w = np.ones((p, 1))

# x = np.arrange (start=1, stop =11, step=1)
err = torch.randn((n, 1))
w = torch.from_numpy(w)

N = 1000 #number of x_testdata observations

x_testdata = torch.rand((N, p), dtype=torch.float64)  # uniform distribution on (0,1) of size N*p
y_testdata = torch.matmul(x_testdata, w) + torch.randn((N, 1))

# Saving the randomly generated data into a text file
with open("data.txt", "wb") as output:
    pickler = pickle.Pickler(output)
    pickler.dump(x_data)
    pickle.dump(err, output, protocol=pickle.HIGHEST_PROTOCOL)
    torch.save(err, 'data.txt')
    pickle.dump(x_testdata, output, protocol=pickle.HIGHEST_PROTOCOL)
    torch.save(x_testdata, 'data.txt')
    pickle.dump(y_testdata, output, protocol=pickle.HIGHEST_PROTOCOL)
    torch.save(y_testdata, 'data.txt')
    # pickler.dump(x_testdata)
    # pickler.dump(y_testdata)

output.close()

with open("data.txt", "rb") as input:
    print(pickle.load(input))  # printing first object in this file
    print(pickle.loads(input.read()))  # printing 2nd object in this file
    print(pickle.loads(input.read()))  # printing 3rd object in this file
    print(pickle.loads(input.read()))  # printing 4th object in this file

input.close()

Can someone help me to solve this error please?

Ashwani Agarwal
  • 1,279
  • 9
  • 30
Eileen
  • 11
  • 1
  • 1
    Does this answer your question? [How can you Pickle multiple objects in one file?](https://stackoverflow.com/questions/42350635/how-can-you-pickle-multiple-objects-in-one-file) – r.ook Jun 11 '20 at 16:47
  • Does this answer your question? [Saving and loading multiple objects in pickle file?](https://stackoverflow.com/questions/20716812/saving-and-loading-multiple-objects-in-pickle-file) The top answer addresses the possibility of calling `load` (not `loads`!) more than once. – Davis Herring Jun 12 '20 at 03:50

0 Answers0