3

Using the double star syntax in function definition, we obtain a regular dictionary. The problem is that it loose the user input order. Sometimes, we could want to know in which order keyword arguments where passed to the function.

Since usually a function call do not involved many arguments, I don't think it is a problem of performance so I wonder why the default is not to maintain the order.

I know we can use:

from collections import Ordereddict
def my_func(kwargs):
    print kwargs
my_func(Ordereddict(a=1, b=42))

But it is less concise than:

def my_func(**kwargs):
    print kwargs
my_func(a=1, b=42)

[EDIT 1]:

1) I thought there where 2 cases:

  • I need to know the order, this behaviour is known by the user through the documentation.
  • I do not need the order, so I do not care if it is ordered or not.

I did not thought that even if the user know it use the order, he could use:

a = dict(a=1, b=42)
my_func(**a)

Because he did not know that a dict is not ordered (even if he should know)

2) I thought that the overhead would not be huge in case of a few arguments, so the benefits of having a new possibility to manage arguments would be superior to this downside.

But it seems (from Joe's answer) that the overhead is not negligible.

[EDIT 2]:

It seems that the PEP 0468 -- Preserving the order of **kwargs in a function is going in this direction.

PhML
  • 1,190
  • 1
  • 11
  • 20
  • I think this is more a case of *why* than *why not* - needing order on keyword arguments is a rare case, and ordered dictionaries are not as core as `dict`s, so it just wasn't done that way. – Gareth Latty Aug 06 '13 at 10:16
  • @Lattyware I would say the need is rare because it's not available. If I could rely on the order of keyword arguments being preserved I would use the feature often. – piokuc Aug 06 '13 at 10:36
  • @piokuc Really? Well, either way, it would muddy the semantics of keyword arguments a lot - see my answer. – Gareth Latty Aug 06 '13 at 10:47
  • The `**kwargs` since the python 3.6 is an `OrderedDict`, therefore the keyword argument order is being preserved. Please see here: http://stackoverflow.com/questions/8977594/in-python-what-determines-the-order-while-iterating-through-kwargs/41634018#41634018 – afxentios Jan 13 '17 at 11:53

4 Answers4

7

Because dictionaries are not ordered by definition. I think it really is that simple. The point of kwargs is to take care of exactly those formal parameters which are not ordered. If you did know the order then you could receive them as 'normal' parameters or *args.

Here is a dictionary definition.

CPython implementation detail: Keys and values are listed in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary’s history of insertions and deletions.

http://docs.python.org/2/library/stdtypes.html#dict

Python's dictionaries are central to the way the whole language works, so they are highly optimised. Adding ordering would impact performance and require more storage and processing overhead.

You may have a case where that's not true, but I think that's more exceptional than common. Adding a feature 'just in case' for a very hot code path is not a sensible design decision.

EDIT:

Just FYI

>>> timeit.timeit(stmt="z = dict(x)", setup='x = ((("one", "two"), ("three", "four"), ("five", "six")))', number=1000000)
1.6569631099700928

>>> timeit.timeit(stmt="z = OrderedDict(x)", setup='from collections import OrderedDict; x = ((("one", "two"), ("three", "four"), ("five", "six")))', number=1000000)
31.618864059448242

That's about a 30x speed difference in constructing a smallish 'normal' size dictionary. OrderedDict is part of the standard library, so I don't imagine there's much more performance that can be squeezed out of it.

Joe
  • 46,419
  • 33
  • 155
  • 245
  • That's a terrible way to time something - the `timeit` module exists for a reason. – Gareth Latty Aug 06 '13 at 10:31
  • It's quick and dirty and measures an elapsed time difference. "Terrible" though? There's a factor of 30. You should contribute your answer to http://stackoverflow.com/questions/3426870/calculating-time-difference – Joe Aug 06 '13 at 10:38
  • "Because dictionaries are not ordered by definition." - I think you confusing dictionaries with hashtables. – piokuc Aug 06 '13 at 10:40
  • @piokuc read the definition here: http://docs.python.org/2/library/stdtypes.html#dict . It says "Keys and values are listed in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary’s history of insertions and deletions." – Joe Aug 06 '13 at 10:42
  • There are many ways to implement a dictionary. The simplest is a list of pairs, which is actually pretty good data structure for small number of items, which is normally a case with keyword arguments. So, in my opinion, your test is useless - the language feature could be implemented without using the current implementation of OrderedDict. – piokuc Aug 06 '13 at 10:44
  • @Joe I'm not talking about dictionary as implemented in Python, but Dictionary as an abstract data structure. – piokuc Aug 06 '13 at 10:45
  • @Joe That's not asking how to test a code snippet - that question is probably talking about adding a timer to a webpage for generation time, say, which has different aims and requirements. `timeit` is the best way to time snippets accurately. – Gareth Latty Aug 06 '13 at 10:46
  • @piokuc In his answer, Joe was clearly talking about Python dicts. – Gareth Latty Aug 06 '13 at 10:46
  • @piokuc this question is about the implementation of Python. I don't know why you're talking about not-Python. – Joe Aug 06 '13 at 10:48
  • @Lattyware OK. What I say is that keyword arguments could be implemented so they would preserve the order, and not necessarily be slower than they are now. – piokuc Aug 06 '13 at 10:48
  • @Lattyware updated to use `timeit`. Not disputing the use of `timeit` or the different types of benchmark metrics, but the result is basically the same. – Joe Aug 06 '13 at 10:49
  • 1
    Just to remind us that the question we are talking about is "Why Python’s function call semantics pass-in keyword arguments are not ordered?". I.e. "Why is Python implemented as it is". – Joe Aug 06 '13 at 10:50
  • @Joe In this case, `timeit` produced the same results - that will not always be the case, and you would never know until you try them both - hence one should always use `timeit` - it covers the edge cases. Yes, they are rare, by why reinvent the wheel and not do it properly? – Gareth Latty Aug 06 '13 at 10:54
  • @Lattyware because I was being lazy and was trying to demonstrate an order-of-magnitude difference. As I say, I'm not disputing different ways of timing things. I'll delete the time-elapsed version from my answer. – Joe Aug 06 '13 at 10:55
  • 1
    Hash Table is the fastest implementation of Dictionaries AFAIK. If you know an algorithm for implementing Ordered Dictionaries which is faster (or in a worth-way slower, considering it's use case percentage in functions) than Hash Table, please tell us. – saeedgnu Aug 06 '13 at 11:01
  • Re: "this question is about the implementation of Python. I don't know why you're talking about not-Python." this is about possible, alternative implementation of Python, so we don't have to talk about OrderedDictionary vs. standard Python dictionary, we are free to talk about other possible implementations of dictionaries – piokuc Aug 06 '13 at 12:19
3

As a counter-argument, here is an example of the complicated semantics this would cause. There are a couple of cases here:

  • The function always gets an unordered dictionary.
  • The function always gets an ordered dictionary - given this, we don't know if the order has any meaning, as if the user passes in an unordered data structure, the order will be arbitrary, while the data type implies order.
  • The function gets whatever is passed in - this seems ideal, but it's not that simple.

What about the case of some_func(a=1, b=2, **unordered_dict)? There is implicit ordering in the original keyword arguments, but then the dict is unordered. There is no clear choice here between ordered or not.

Given this, I'd say that ordering the keyword arguments wouldn't be useful, as it would be impossible to tell if the order is just an arbitrary one. This would cloud the semantics of function calling.

Given that, any benefit gained by making this a part of calling is lost - instead, just expect an OrderedDict as an argument.

Gareth Latty
  • 86,389
  • 17
  • 178
  • 183
0

If your function's arguments are so correlated that both name and order matter, consider using a specific data structure or define a class to hold them. Chances are, you'll want them together in other places in your code, and possibly define other functions/methods that use them.

Nicola Musatti
  • 17,834
  • 2
  • 46
  • 55
0

Retrieving the order of key-word arguments passed via **kwargs would be extremely useful in the particular project I am working on. It is about making a kind of n-d numpy array with meaningful dimensions (right now called dimarray), particularly useful for geophysical data handling.

I have posted a developed question with examples here:

How to retrieve the original order of key-word arguments passed to a function call?

Community
  • 1
  • 1
Mahé
  • 445
  • 4
  • 9
  • This isn't an answer, and so should be removed. SO hews to a (reasonably) strict question/answer format. – DSM Dec 01 '13 at 16:53