Truly Private Variables in Python 3

Question

So I know the way to make a variable "private" in python like this:

class Foo:
    def __init__(self):
        self.__private = 'bar'

This "works" and doesn't, as shown below:

foo = Foo()
'__private' in vars(foo) #False
'_Foo__private' in vars(foo) #True

Now, I understand this is the way to make private variables in python and I like this way. It allows you to mangle names so that no subclasses accidentally override this (because it begins with the class's name), and that nobody will accidentally use it. It also gives you the power to change the private variables if you know what you are doing. Also, it is the best way to do it, because truly private variables are impossible.

Or so I thought.

Recently, I was reading PEP 8 and I saw this line:

We don't use the term "private" here, since no attribute is really private in Python (without a generally unnecessary amount of work).

This quote is found in the Designing for Inheritance section of PEP 8.

Note the phrase "without a generally unnecessary amount of work". I am now sure that there must be a way to get truly private variables in python. How would I do that?

I have tried overriding __getattribute__, but the problem is that there is no way to tell if the call is coming from inside the class or not (that I am aware of).

Also, the __dict__ attribute is annoying when trying to do this because it holds references to all instance variables.

I also thought of metaclasses, but those seem to have the same problems as __getattribute__.

Thoughts?

---

Note: I understand that any way to make truly private variables in python should never be done in productive code. I just want to know how it could be done.

Answer 1

I have tried overriding getattribute, but the problem is that there is no way to tell if the call is coming from inside the class or not (that I am aware of).

You can use the inspect module to find the name and module of the calling function, which you could compare against a whitelist.

But inspect also has getattr_static, which can bypass any __getattribute__.

---

Nothing is truly private in Python. There are ways to make access difficult, but there are always ways around those ways.

The only solution then, is outside of the current Python interpreter. You could use a foreign function interface to some other more secure language or a remote procedure call (e.g. xmlrpc) to the same or to another Python interpreter running in a subprocess, or even one running as a different user with different permissions. The private variable and all the functions allowed to access it will live outside the current interpreter. Then there's no way to inspect it.

This type of privilege separation is even one of the stated use cases for the Pyro RPC library.

Answer 2

You can get nearly the same effect without the fancy inspection by using closures instead of attributes.

class Foo:
    def __init__(self):
        private = 'bar'
        def print_private():
            print(private)
        self.print_private = print_private

foo = Foo()
foo.print_private()  # works
foo.private  # kaboom

Of course, inspect can see into closures too.

Answer 3

The reason why Python has no private attributes is that we can't tell whether it is inside or outside a class. They share the same process of attribute access. self.private is exactly the obj.private. So, if we prevent from obj.private, self.private is also prevented. The only way to differ them is to give different name and make the obj.private be the proxy of self._private by @property or data descriptor and believe that people using it are all adults.

Anyway, I'd like to share the concept of data descriptor which could make NEARLY private attributes by adding a layer of attribute proxy (As I said, this would prevent the access from 'inside' the class):

class Private:
    def __init__(self, attribute):
        self.attribute = attribute

    def __get__(self, obj, type=None):
        raise AttributeError("'{}' object has no attribute '{}'".format(obj, self.attribute))

    def __set__(self, obj, value):
        obj.__dict__[self.attribute] = value

class YourClass:
    private = Private('private')

    def __init__(self):
        self.private = 10
        print(self.private)  # Raise AttributeError

Use double underlines or change __getattribute__are both bad practices, especially the latter, may cause disasters.

Answer 4

Well after looking at this answer about the inspect module, I (kind of) have done it!

class Foo:
    def __init__(self, private):
        self.private = private

    def __getattribute__(self, attr):
        import inspect
        frame = inspect.currentframe()
        try:
            back_self = frame.f_back.__self__
            if not back_self == self: #is it inside the class?
                ban = ('private', '__dict__') #all private vars, ban __dict__ for no loopholes
                if attr in ban:
                    msg = 'Foo object has no attribute {!r}'
                    raise AttributeError(msg.format(attr))
        finally:
            del frame
        return super().__getattribute__(attr)

    def print_private(self):
        print(self.private) #access in the class!


foo = Foo('hi')
foo.print_private() #output: hi
foo.private #makes an error

Well, almost. inspect can also be used to find the value, too. This is very close, though. It allows object.attr inside the class but creates an error if called from the outside. This is probably as close as one can get.

Answer 5

something I like to do, though it's not exactly 100% private, is to use closures in methods to R/W normally inaccessible attributes as member_descriptor objects:

def privateNS():

    class MyObject(object):
        __slots__ = ['private'] # name doesn't matter

        def __new__(cls, value): # only sets inst.private on new instance creation
            inst = object.__new__(cls)

            setprivate(inst, value)

            return inst

        # __init__ is not needed, and can't be used here to set inst.private

        def showprivate(inst):
            return getprivate(inst)

    dsc = MyObject.private # get descriptor
    getprivate = dsc.__get__
    setprivate = dsc.__set__
    del MyObject.private # revoke normal access

    return MyObject

MyObject = privateNS()
del privateNS

inst = MyObject( 20 )
print( inst.showprivate() ) # 20

note that the inst.private name does not exist and will raise an AttributeError if referenced.
but the member descriptor itself does exist, and is bound to the class.

but like I said, it's not 100% private...
you can access the descriptor methods provided to the class methods through their closures:

>>> inst.showprivate.__closure__[0].cell_contents
<method-wrapper '__get__' of member_descriptor object at 0x00E588A0>

that's the first backdoor, if said method contains __set__ in it's closures.
but if not, the 2nd backdoor is only a tad more complicated:

>>> inst.showprivate.__closure__[0].cell_contents.__self__.__set__( inst, 30 )
>>> inst.showprivate()
30

something that helps though is when using multiple closures, the order of the closure cells is dependent on the current run (like dictionary keys).

sadly though, I can't seem to figure out anything more secure than this...

the problem is as stated in an earlier answer:
attributes can't tell where they're being accessed, and providing that level of functionality through python code always leaves them open because they can always be accessed and changed.

if I'm wrong on this, please comment :)