1

I've been reading about how make Python classes less dynamic, specifically by not allowing users to dynamically create new attributes. I've read that overloadding __setattr__ is a good way to do this , and __slots__ is not the way to go. One post on this last thread actually suggests that __slots__ can break pickling. (Can anyone confirm this?)

However, I was just reading the whatsnew for Python 2.2, and the attribute access section actually suggests using __slots__ for the very purpose of constraining attribute creation, not just for optimization like others have suggested. In terms of Python history, does anyone know what the original intention of __slots__ was? Is constraining variable creation a feature or a bug to be abused? How have people seen __slots__ used in practice? Have many people seen __setattr__ overloaded to restrict attribute creation? Which one is best? If you are more familiar with one method or the other, feel free to post the pros and cons of the method you know. Also, if you have a different way of solving the problem, please share! (And please try not to just repeat the downsides of __slots__ that have been expressed in other threads.)

EDIT: I was hoping to avoid discussion of "why?", but the first answer indicates that this is going to come up, so I'll state it here. In the project in question, we're using these classes to store "configuration information", allowing the user to set attributes on the objects with their (the users') parameters, and then pass the objects off to another part of the program. The objects do more than just store parameters, so a dictionary wouldn't work. We've already had users accidentally type an attribute name wrong, and end up creating a new attribute rather than setting one of the attributes that the program expects. This goes undetected, and so the user thinks they're setting the parameter but doesn't see the expected result. This is confusing for the user, and hard to spot. By constraining attribute creation, an exception will be thrown rather than pass by silently.

EDIT 2, re pickling: These objects will be something that we will want to store in the future, and pickling seems like a good way to do this. If __slots__ is clearly the best solution, we could probably find another way to store them, but pickling would definitely be of value, and should be kept in consideration.

EDIT 3: I should also mention that memory conservation isn't an issue. Very few of these objects will be created, so any memory saved will be negligible (like 10s of kilobytes on a 3-12 GB machine).

Community
  • 1
  • 1
hunse
  • 3,175
  • 20
  • 25
  • 1
    I believe that slots was intended (and still is used) to make a class instance's memory footprint smaller. – mgilson Dec 05 '13 at 17:14
  • 1
    Just use `__slots__`, it does what you need plus conserve the memory, which is good – piokuc Dec 05 '13 at 17:16
  • 1
    @piokuc -- but it breaks pickling and anything else which relies on the object having a defined `__dict__` which isn't completely uncommon. The actual recommendation here would have to be determined by what OP intends to use this class for. Of course, a more fair question would be why does OP want to constrain attribute creation in the first place? We're all consenting adults here ... – mgilson Dec 05 '13 at 17:18
  • @mgilson I agree it would be useful to know more about the specific class and its purpose, i.e. not every object needs to be pickled – piokuc Dec 05 '13 at 17:21
  • @mgilson I added a description of why I want to constrain attribute creation. As a general question, why do so many people seem to be against this? I realize Python is a dynamic language, but there are few times when I've wanted to add a new attribute to an existing object. Many libraries, for example Numpy, don't allow you to add attributes to their classes (I tried). Maybe I just feel this way because I came from an OOP background. – hunse Dec 05 '13 at 17:33
  • @hunse -- The question isn't why you want to do something like that. The question is why prohibit someone else from doing it if they think it's necessary? Generally, the practice of monkey-patching like this *is* frowned upon as it leads to hard to document code. If there's a good reason to disallow it (optimization in `numpy`'s case), that's fine ... But you shouldn't do it just to do it. As a side note ... What do you mean by the OOP comment. Python has quite a lot of support for OOP ... By that, do you mean that you used to write Java ;-) – mgilson Dec 05 '13 at 17:40
  • In response to your first EDIT, I think that a more pythonic approach would be to validate the input/configuration file prior to constructing the class. That really addresses all your listed needs. The class is still pickleable, the input gets validated so you know the user didn't put in some wrong data. And, you don't have funky/suprising behavior from messing with `__slots__` or `__setattr__`. – mgilson Dec 05 '13 at 17:48

2 Answers2

4

Why are you trying to restrict developers from doing that? Is there any technical reason besides "I don't want them to do it" for it? If not, don't do it.

Anyway, using __slots__ saves memory (no __dict__) so it's the better solution to do it. You won't be able to use it if some of your code needs the object to have a __dict__ though.

If there is a good reason to restrict it (again, are you sure there is one?) or you need to save that little bit of memory, go for __slots__.

After reading your explanation you might want to use __setattr__, possibly combined with __slots__ (you need to store the attribute whitelist somewhere anyway, so you can as well use it to save memory). That way you can display some more helpful information such as which similar attribute are available. A possible implementation could look like this:

class Test(object):
    __slots__ = ('foo', 'bar', 'moo', 'meow', 'foobar')

    def __setattr__(self, name, value):
        try:
            object.__setattr__(self, name, value)
        except AttributeError:
            alts = sorted(self.__slots__, key=lambda x: levenshtein(name, x))
            msg = "object has no attribute '{}', did you mean '{}'?"
            raise AttributeError(msg.format(name, alts[0]))

The levenshtein() I tested it with was implementation #4 from this site. In case of not so smart users you might want to make the error even more verbose and include all close matches instead of just the first one.

You can further improve the code by creating a mixin class containing just the __setattr__ method. That way you can keep the code out of your real classes and also have a custom __setattr__ if necessary (just use super(...).__setattr__(...) in the mixin instead of object.__setattr__)

ThiefMaster
  • 310,957
  • 84
  • 592
  • 636
  • 1
    +1 for repeatedly asking OP to think about whether there really is a need to restrict attribute creation :) – mgilson Dec 05 '13 at 17:20
  • I've added a description of the problem to my post. The long and short of it is: yes, I do have a good reason to do this. I should also note that a lot of our users are not developers, but people with math and science backgrounds who have limited programming experience. – hunse Dec 05 '13 at 17:29
  • I like the idea of providing close name matches; that would be great for typos or adding an 's', etc. I might end up not using slots to store the attribute list, just because it would break Pickling, but you're right, it does need to be stored somewhere. – hunse Dec 05 '13 at 17:57
  • @hunse where this is from 3 years, I would just like to emphasis that you can easily fix the broken pickle using `__setstate__` and `__getstate__`. I created a [gist](https://gist.github.com/guyarad/a95474daf0542b3344e30c534afe6450) to demonstrate. – Guy Sep 08 '16 at 13:35
1

My suggestion for your use case is to use __setattr__ and issue a warning when the attribute name is not recognized using Python's warnings module.

kindall
  • 178,883
  • 35
  • 278
  • 309
  • My suggestion is to validate *before* creating the class and leave `__setattr__` out of it. `warning` is a good call though. – mgilson Dec 05 '13 at 17:49
  • Warnings are a great idea, especially because they can be treated as errors or warnings or even suppressed depending on user preferences. I don't know much about the `warnings` module, as we've used a logger for warnings in the past, but I'm definitely going to look into it now. – hunse Dec 05 '13 at 19:51