2

  • I have read that I can read pickles that were dumped in python 2.7 in python 3 using

    content = pickle.load(o, encoding='latin1')

  • Obviously, I can read pickles that were dumped in python 3 using

    content = pickle.load(o)

My problem is, I can't know the source of my pickle. It could be either one.

How can I test which type of pickle I am trying to read in order to use the correct method?

Gulzar
  • 23,452
  • 27
  • 113
  • 201
  • 1
    Wouldn't `pickle.load(o)` raise an exception if `encoding='latin1'` was expected? in that case you can use `try-except` with the correct exception type – DeepSpace Jan 02 '20 at 11:33
  • Wouldn't it possible to use the [Pickle protocol versions](https://docs.python.org/3/library/pickle.html#data-stream-format) in combination with [pickletools](https://docs.python.org/3/library/pickletools.html#module-pickletools) to identify in which Python version the file was created. – Maurice Meyer Jan 02 '20 at 12:16

2 Answers2

0

inspired by @DeepSpace:

def load(path):
    print(f"loading pkl: {os.path.abspath(path)}")
    assert os.path.isfile(path)
    try:
        with open(path, "rb") as f:
            content = pickle.load(f)
    except UnicodeDecodeError:  # pickle was created with python 2.7
        with open(path, "rb") as f:
            content = pickle.load(f, encoding='latin1')

    return content
Gulzar
  • 23,452
  • 27
  • 113
  • 201
-1

use if else: try to do normal load without encoding and do some reads, if success proceed else load with encoding='latin1'

hariram manohar
  • 57
  • 1
  • 7