45

I'm trying to sort a list of objects using

my_list.sort(key=operator.attrgetter(attr_name))

but if any of the list items has attr = None instead of attr = 'whatever',

then I get a TypeError: unorderable types: NoneType() < str()

In Py2 it wasn't a problem. How do I handle this in Py3?

jsalonen
  • 29,593
  • 15
  • 91
  • 109
AlexVhr
  • 2,014
  • 1
  • 20
  • 30

5 Answers5

40

The ordering comparison operators are stricter about types in Python 3, as described here:

The ordering comparison operators (<, <=, >=, >) raise a TypeError exception when the operands don’t have a meaningful natural ordering.

Python 2 sorts None before any string (even empty string):

>>> None < None
False

>>> None < "abc"
True

>>> None < ""
True

In Python 3 any attempts at ordering NoneType instances result in an exception:

>>> None < "abc"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: NoneType() < str()

The quickest fix I can think of is to explicitly map None instances into something orderable like "":

my_list_sortable = [(x or "") for x in my_list]

If you want to sort your data while keeping it intact, just give sort a customized key method:

def nonesorter(a):
    if not a:
        return ""
    return a

my_list.sort(key=nonesorter)
jsalonen
  • 29,593
  • 15
  • 91
  • 109
  • Yes, I've allready noticed that. What I'm asking for is a solution. If `attr` was of some `custom_type`, I whould just override `custom_type.__lt__()`, but `None` is a built in. – AlexVhr Oct 19 '12 at 09:59
  • 1
    It wasn't me - but one improvement would be to use `""` instead of `0` as the `None` substitute since the OP appears to be comparing strings. – Tim Pietzcker Oct 19 '12 at 10:08
  • But that will change the actual data inside the objects being sorted, will it not? – AlexVhr Oct 19 '12 at 10:09
  • @AlexVhr: check the edit again for a solution that doesn't change your list. – jsalonen Oct 19 '12 at 10:24
  • 1
    It works, but requires objects to have comparison methods, like `__lt__()` Otherwise it couses `TypeError: unorderable types: MyObj() < MyObj()` – AlexVhr Oct 19 '12 at 10:59
34

For a general solution, you can define an object that compares less than any other object:

from functools import total_ordering

@total_ordering
class MinType(object):
    def __le__(self, other):
        return True

    def __eq__(self, other):
        return (self is other)

Min = MinType()

Then use a sort key that substitutes Min for any None values in the list

mylist.sort(key=lambda x: Min if x is None else x)
augurar
  • 12,081
  • 6
  • 50
  • 65
  • 1
    Not really that general: `TypeError: unorderable types: type() < tuple()` If your list contains tuples. – letmaik Feb 10 '15 at 22:16
  • 1
    @neo You made a mistake in your test. Obviously using this class won't prevent other, unrelated type errors. Notice that neither of the arguments in the type error were of type `MinType`. – augurar Feb 10 '15 at 23:31
  • Ah, sorry, I had used `MinType` (which is a `type`) instead of `Min`. – letmaik Feb 11 '15 at 07:45
  • 1
    Much better than the other solution, IMHO, and very elegant. The kind of thing I'd never have found alone. Thanks a lot ! – JulienD Sep 29 '15 at 10:00
  • 1
    Be aware that this might lead to inconsistent ordering if you have more than one of these objects competing to be the smallest. A safe way would be to have `__le__` compare `id(self)` and `id(other)` when the two objects are both instances of `MinType`. – T Tse May 31 '18 at 01:08
  • 1
    @ShioT The intent is to use the `Min` object as a singleton. – augurar Jun 18 '18 at 20:07
  • @augurar Hmmm.. Sorry sometimes my Java instinct still kicks in and tell me to defensively make the `__init__` method throw an exception and have a `getInstance` class method / static method instead – T Tse Jun 21 '18 at 01:41
  • Yeah, this is more of a "sketch" of the approach, but you could certainly make it more robust in various ways. – augurar Jun 21 '18 at 20:33
  • How do set an attribute this solution compares objects by? Like, I want to sort instances of a class `Person` by age. The standard way would be `my_list.sort(key=operator.attrgetter('age'))`. How do I do that with this solution? – AlexVhr Jul 04 '20 at 14:28
  • Why is the total_ordering annotation required? Can the lambda be abbreviated to `key=lambda x: x or Min` as suggested by an answer below? – chrisinmtown Apr 06 '22 at 02:24
12

The solutions proposed here work, but this could be shortened further:

mylist.sort(key=lambda x: x or 0)

In essence, we can treat None as if it had value 0.

E.g.:

>>> mylist = [3, 1, None, None, 2, 0]
>>> mylist.sort(key=lambda x: x or 0)
>>> mylist
[None, None, 0, 1, 2, 3]
fralau
  • 3,279
  • 3
  • 28
  • 41
  • doesn't work if you're sorting numerical lists with possible None, eg `[3,1,None,2]` – alancalvitti Nov 01 '19 at 15:52
  • That puts `None` arbitrarily in the middle of the list if there's negatives and nonnegatives, eg `[3, 1, None, None, -2, 0]` – alancalvitti Nov 01 '19 at 18:52
  • Yes. The question was about removing the error, and that removed the error. Since it did not specify what one was supposed to do with the None, I left the Nones there. – fralau Nov 02 '19 at 06:18
  • Even if you don't care about None you should watch out for other 'falsy' values too, for instance, 0, which if you're doing a mapping to sort would be tempting to start with. – i30817 Sep 06 '22 at 13:45
4

A general solution to deal with None independent of the types to be sorted:

my_list.sort(key=lambda x: (x is not None, x))

putting None values first.

Note that: my_list.sort(key=lambda x: (x is None, x)) puts None values last.

eci
  • 2,294
  • 20
  • 18
3

Since there are other things besides None that are not comparable to a string (ints and lists, for starters), here is a more robust solution to the general problem:

my_list.sort(key=lambda x: x if isinstance(x, str) else "")

This will let strings and any type derived from str to compare as themselves, and bin everything else with the empty string. Or substitute a different default default key if you prefer, e.g. "ZZZZ" or chr(sys.maxunicode) to make such elements sort at the end.

alexis
  • 48,685
  • 16
  • 101
  • 161
  • 1
    how would you sort numerical lists with possible nones, (see comment to fralau) – alancalvitti Nov 01 '19 at 15:53
  • In response to alancalvitti's 2-year old question - fralau took advantage of None being treated as False in the "or" comparison statement, which is an effective and simple way to do it. If you wish to be more explicit on numerical lists (presuming only values > 0 ): `my_list.sort(key=lambda x: 0 if x is None else x)` – Nate Wanner Nov 10 '21 at 14:36