Can a python pickle object be checked for malicious code before unpickling?

Question

Is there any way to test a pickle file to see if it loads a function or class during unpickling?

This gives a good summary of how to stop loading of selected functions: https://docs.python.org/3/library/pickle.html#restricting-globals

I assume it could be used to check if there is function loading at all, by simply blocking all function loading and getting an error message.

But is there a way to write a function that will simply say: there is only text data in this pickled object and no function loading?

I can't say I know which builtins are safe!

PickleMagic is perhaps what you are looking for. It claims to load pickle files safely ?? But I have not tested this myself: Maybe worth looking into: https://github.com/CensoredUsername/picklemagic — ScottC, Jan 17 '23 at 22:42
Yes, that is my problem: can't tell if picklemagic is bullet proof, though the python doc above seems to indicate a way that makes pickle bullet proof safe. — wgw, Jan 18 '23 at 00:43

score 3 · Accepted Answer · answered Jan 17 '23 at 22:11

Basically no, there is truly no way. There is a lot written about this. You can only use pickle if you trust the source, and you get the pickle directly from the source. Any safety measures you perform are not sufficiënt to protect against mallicious attempts whatsoever.

https://medium.com/ochrona/python-pickle-is-notoriously-insecure-d6651f1974c9 https://nedbatchelder.com/blog/202006/pickles_nine_flaws.html etcetera.

I use it sometimes, but then most of the times after I have had a phonecall with a colleague and shared a pickled file. But more often, I use it for myself on my local environment to store data. Still, this is not the preferred way, but it's fast.

So, when in doubt. Do not use pickle.

Picklemagic claims to fix the security issues. I'm not sure I can tell if it truly does. https://github.com/CensoredUsername/picklemagic — wgw, Jan 17 '23 at 22:45
I will go through the doc -- thanks! The python doc seems to suggest that there is a way to make it safe, but clearly I need to read more! — wgw, Jan 18 '23 at 00:45

score 0 · Answer 2 · answered Jan 18 '23 at 19:14

Thanks for the answers!

Straight pickle is too prone to security issues.

Picklemagic claims to fix the security issues; it looks like it does, but I can't quite confirm that: http://github.com/CensoredUsername/picklemagic

But https://medium.com/ochrona/python-pickle-is-notoriously-insecure-d6651f1974c9 suggest that there is no safe wrapper (picklemagic has been around for 8 years, and the article dates from 2021; so picklemagic was not considered?)

The only surefire way protect against Pickle Bombs is not to use pickle directly. Unfortunately unlike other unsafe standard library packages there are no safe wrappers or drop-in alternatives available for pickle, like defusedxml for xml or tarsafe for tarfile. Further there’s not a great way to inspect a pickle prior to unpickling or to block unsafe function calls invoked by REDUCE.

The 3.10 docs does offer a wrapper for blocking unauthorized execution of a function. https://docs.python.org/3.10/tutorial/controlflow.html

It does not say which builtins are safe. If os is removed, are the others safe? Still, if it is clear what is supposed to be in the pickled object, it may be easy enough to restrict execution.

import builtins
import io
import pickle

safe_builtins = {
    'range',
    'complex',
    'set',
    'frozenset',
    'slice',
}

class RestrictedUnpickler(pickle.Unpickler):

    def find_class(self, module, name):
        # Only allow safe classes from builtins.
        if module == "builtins" and name in safe_builtins:
            return getattr(builtins, name)
        # Forbid everything else.
        raise pickle.UnpicklingError("global '%s.%s' is forbidden" %
                                     (module, name))

def restricted_loads(s):
    """Helper function analogous to pickle.loads()."""
    return RestrictedUnpickler(io.BytesIO(s)).load()

A good (better!) discussion is here: https://stackoverflow.com/questions/25353753/python-can-i-safely-unpickle-untrusted-data?rq=1 — wgw, Jan 23 '23 at 04:22

Can a python pickle object be checked for malicious code before unpickling?

2 Answers2