What is "pickle" actually used for?

Question

I'm a beginner in coding and my project now involves image classification. In the below code from the deep learning class, there is the function maybe_pickle but I don't know what does "pickle" do and how helpful it is though I have tried searching for some answers.

def maybe_pickle(data_folders, min_num_images_per_class, force=False):


dataset_names = []
  for folder in data_folders:
    set_filename = folder + '.pickle'
    dataset_names.append(set_filename)
    if os.path.exists(set_filename) and not force:
      # You may override by setting force=True.
      print('%s already present - Skipping pickling.' % set_filename)
    else:
      print('Pickling %s.' % set_filename)
      dataset = load_letter(folder, min_num_images_per_class)
      try:
        with open(set_filename, 'wb') as f:
          pickle.dump(dataset, f, pickle.HIGHEST_PROTOCOL)
      except Exception as e:
        print('Unable to save data to', set_filename, ':', e)

  return dataset_names

train_datasets = maybe_pickle(train_folders, 45000)
test_datasets = maybe_pickle(test_folders, 1800)
# import pickle

# Create a list
test_values = ['test value','test value 2','test value 3']


file_Name = "testfile"
# Open the file for writing
fileObject = open(file_Name,'wb') 

# This writes the object a to the
# file named 'testfile'
pickle.dump(test_values, fileObject)   

# Then we close the fileObject
fileObject.close()

# We then open the file for reading
fileObject = open(file_Name,'r')  

# And the object from the file into var b
test_values_loaded = pickle.load(fileObject)

Pickling saves objects to file so they can be loaded again later. It's just object serialization. If you were making a game, you might pickle the player's game save so it could be loaded later. — Carcigenicate, Aug 24 '17 at 11:51
Possible duplicate of [Understanding Pickling in Python](https://stackoverflow.com/questions/7501947/understanding-pickling-in-python) — Chris_Rands, Aug 24 '17 at 11:58
Also see: https://stackoverflow.com/questions/8968884/python-serialization-why-pickle and https://stackoverflow.com/questions/21752259/python-why-pickle — Chris_Rands, Aug 24 '17 at 11:59

score 0 · Answer 1 · answered Aug 24 '17 at 11:54

First check the official pickle documentation. It takes a Python object which can be serialized and turns it into a bytestream which you can save to disk (or the inverse, start from a file and turn this back into a Python object. This is often done if you want to save state between run or if you want to cache the result of long running operations (e.g. maybe_* like functions typically serve this purpose).

What is "pickle" actually used for?

1 Answers1