How to design a library public api avoiding to expose internals?

Question

I am studying python. I am trying to understand how to design a library that exposes a public api. I want avoid to expose internal methods that could change in future. I am looking for a simple and pythonic way to do it. I have a library that contains a bunch of classes. Some methods of those classes are used internally among classes. I don't want to expose those methods to the client code.

Suppose that my library (f.e. mylib) contains a class C with two methods a C.public() method thought to be used from client code and C.internal() method used to do some work into the library code. I want to commit myself to the public api (C.public()) but I am expecting to change the C.internal() method in future, for example adding or removing parameters.

The following code illustrates my question:

mylib/c.py:

class C:
    def public(self):
        pass

    def internal(self):
        pass

mylib/f.py:

class F:
    def build():
        c = C()
        c.internal()
        return c

mylib/__init__.py:

from mylib.c import C
from mylib.f import F

client/client.py:

import mylib
f = mylib.F()
c = f.build()
c.public()
c.internal()  # I wish to hide this from client code

I have thought the following solutions:

• document only public api, warning user in documentation to don't use private library api. Live in peace hoping that clients will use only public api. If the next library version breaks client code is the client fault:).

• use some form of naming convention, f.e. prefix each method with "_", (it is reserved for protected methods and raises a warning into ide), perhaps I can use other prefixes.

• use objects composition to hide internal methods. For example the library could return to the clients only PC object that embeds C objects.

mylib/pc.py:

class PC:
    def __init__(self, c):
        self.__c__

    def public(self):
        self.__cc__.public()

But this looks a little contrived.

Any suggestion is appreciated :-)

Update

It was suggested that this question is duplicated of Does Python have “private” variables in classes?

It is similar question but I is a bit different about scope. My scope is a library not a single class. I am wondering if there is some convention about marking (or forcing) which are the public methods/classes/functions of a library. For example I use the __init__.py to export the public classes or functions. I am wondering if there is some convention about exporting class methods or if i can rely only on documentation. I know I can use "_" prefix for marking protected methods. As best as I know protected method are method that can be used in class hierarchy.

I have found a question about marking public method with a decorator @api Sphinx Public API documentation but it was about 3 years ago. There is commonly accepted solution, so if someone are reading my code understand what are methods intended to be library public api, and methods intended to be used internally in the library? Hope I have clarified my questions. Thanks all!

Answer 1

You cannot really hide methods and attributes of objects. If you want to be sure that your internal methods are not exposed, wrapping is the way to go:

class PublicC:
    def __init__(self):
        self._c = C()

    def public(self):
        self._c.public()

Double underscore as a prefix is usually discouraged as far as I know to prevent collision with python internals.

What is discouraged are __myvar__ names with double-underscore prefix+suffix ...this naming style is used by many python internals and should be avoided -- Anentropic

If you prefer subclassing, you could overwrite internal methods and raise Errors:

class PublicC(C):
    def internal(self):
        raise Exception('This is a private method')

If you want to use some python magic, you can have a look at __getattribute__. Here you can check what your user is trying to retrieve (a function or an attribute) and raise AttributeError if the client wants to go for an internal/blacklisted method.

class C(object):
    def public(self):
        print "i am a public method"

    def internal(self):
        print "i should not be exposed"

class PublicC(C):
    blacklist = ['internal']

    def __getattribute__(self, name):
        if name in PublicC.blacklist:
            raise AttributeError("{} is internal".format(name))
        else: 
            return super(C, self).__getattribute__(name) 

c = PublicC()
c.public()
c.internal()

# --- output ---

i am a public method
Traceback (most recent call last):
  File "covering.py", line 19, in <module>
    c.internal()
  File "covering.py", line 13, in __getattribute__
    raise AttributeError("{} is internal".format(name))
AttributeError: internal is internal

I assume this causes the least code overhead but also requires some maintenance. You could also reverse the check and whitelist methods.

...
whitelist = ['public']
def __getattribute__(self, name):
    if name not in PublicC.whitelist:
...

This might be better for your case since the whitelist will probably not change as often as the blacklist.

Eventually, it is up to you. As you said yourself: It's all about documentation.

Another remark:

Maybe you also want to reconsider your class structure. You already have a factory class F for C. Let F have all the internal methods.

class F:
    def build(self):
        c = C()
        self._internal(c)
        return c

    def _internal(self, c):
        # work with c

In this case you do not have to wrap or subclass anything. If there are no hard design constraints to render this impossible, I would recommend this approach.

Answer 2

I have thought the following solutions:

• document only public api, warning user in documentation to don't use private library api. Live in peace hoping that clients will use only public api. If the next library version breaks client code is the client fault:).

• use some form of naming convention, f.e. prefix each method with "_", (it is reserved for protected methods and raises a warning into ide), perhaps I can use other prefixes.

• use objects composition to hide internal methods. For example the library could return to the clients only PC object that embeds C objects.

You got it pretty right with the first two points.

The Pythonic way is to name internal methods starting with single underscore '_', this way all Python developers know that this method is there, but it's use is discouraged and won't use it. (Until they decide to do some monkey-patching, but you shouldn't care for this scenario.) For newbie developers you might want to mention explicitly about not using methods starting with underscore. Also, just don't provide public documentation for your "private" methods, use it for internal reference only.

You might want to take a look at "name mangling", but it's less common.

Hiding internals with object composition or methods like __getattribute__ and etc. is generally discouraged in Python.

You might want to look at source code of some popular libraries to see how they manage this, e.g. Django, Twisted, etc.