django - why is the request.POST object immutable?

Question

As the title asks, why did the Django guys decide to implement the request.POST object with a querydict (which, of course, in turn, makes the whole thing immutable?)

I know you can mutify it by making a copy of the post data

post = request.POST.copy()

but why do this? Surely it would be simpler just to allow the thing to be mutable anyway? Or is it being used for some other reason too which might cause issue?

Why do you want it to be mutable? You can take the data from it and use/modify it in your view. By adding data to it, you could create the impression that `request.POST` has been submitted with more data than it actually has been. — Simeon Visser, Sep 26 '12 at 22:25
It isn't that I *want* it to be mutable. No more than, say, I'd want ice cream to be cold. In the case of ice cream though, if it *isn't* cold it melts and then you get scolded for making a big ol' mess. But with the request.POST object... I mean, if I'm going to screw up my code, I'm going to screw it up. I wasn't aware there was an endemic of developers adding data to POST objects and Causing Problems, so it seems like an odd thing to target to "fix". — bharal, Sep 26 '12 at 22:31
This came up sporadically for me because my client sometimes submitted JSON data (mutable) and sometimes URL Form Encoded (immutable) messages. — owenfi, Jan 09 '15 at 03:16
For non-English speakers, "mutify" isn't a word - the correct phrase is "you can mutate it" or "you can modify it". There's also no need to gender the developers - you could use "Django team" or "core devs" rather than "guys". — alexmuller, Nov 13 '17 at 11:38

score 136 · Accepted Answer · edited May 23 '17 at 11:33

136

It's a bit of a mystery, isn't it? Several superficially plausible theories turn out to be wrong on investigation:

So that the POST object doesn't have to implement mutation methods? No: the POST object belongs to the django.http.QueryDict class, which implements a full set of mutation methods including __setitem__, __delitem__, pop and clear. It implements immutability by checking a flag when you call one of the mutation methods. And when you call the copy method you get another QueryDict instance with the mutable flag turned on.
For performance improvement? No: the QueryDict class gains no performance benefit when the mutable flag is turned off.
So that the POST object can be used as a dictionary key? No: QueryDict objects are not hashable.
So that the POST data can be built lazily (without committing to read the whole response), as claimed here? I see no evidence of this in the code: as far as I can tell, the whole of the response is always read, either directly, or via MultiPartParser for multipart responses.
To protect you against programming errors? I've seen this claimed, but I've never seen a good explanation of what these errors are, and how immutability protects you against them.

In any case, POST is not always immutable: when the response is multipart, then POST is mutable. This seems to put the kibosh on most theories you might think of. (Unless this behaviour is an oversight.)

In summary, I can see no clear rationale in Django for the POST object to be immutable for non-multipart requests.

edited May 23 '17 at 11:33

Community

1
1

answered Sep 27 '12 at 10:54

Gareth Rees

64,967
9
133
163

I've noticed tons of rough edges like this in Django. Must've made sense to somebody at some point, though. – Dan Passaro Jan 07 '14 at 17:46
2

I found this in another Stack answer: "And it must be immutable so that it can be built lazily. The copy forces getting all the POST data. Until the copy, it may not all be fetched. Further, for a multi-threaded WSGI server to work reasonably well, it's helpful if this is immutable" – Seaux Apr 30 '14 at 22:39
Ah, sorry, I missed that. I only read the question and the first 2 points of your post. Was just trying to be nice and follow up with info I had found elsewhere. Maybe it use to load lazily, but now doesn't and the QueryDict is code that just needs to be replaced? idk. – Seaux May 01 '14 at 20:04
12

@Seaux you shouldn't read SO answers lazily when you intend to comment on them. ;-) – Chris Wesseling Oct 14 '14 at 13:16
4

@ChrisWesseling I see what you did there – Seaux Oct 15 '14 at 16:01
2

Even better, the querydict is mutable when I send the request suing the django test client. – user1158559 Mar 10 '17 at 15:02

score 88 · Answer 2 · edited May 23 '17 at 12:10

88

If the request was the result of a Django form submission, then it is reasonable for POST being immutable to ensure the integrity of the data between the form submission and the form validation. However, if the request was not sent via a Django form submission, then POST is mutable as there is no form validation.

You can always do something like this: (as per @leo-the-manic's comment)

#  .....
mutable = request.POST._mutable
request.POST._mutable = True
request.POST['some_data'] = 'test data'
request.POST._mutable = mutable
# ......

edited May 23 '17 at 12:10

Community

1
1

answered Jan 21 '14 at 15:25

Val Neekman

17,692
14
63
66

3

@JoshK: I guess the commenter wanted to make POST mutable, and the code snippet in this answer helped. – ShreevatsaR Jun 26 '15 at 19:57
You can add new key,value but you cant change the existing data. – Vamsidhar Muggulla Jun 03 '17 at 07:18
Nice. And I'm sure whoever use this code knows what he or she is doing. – John Pang Sep 20 '18 at 09:21
@VamsidharMuggulla Both adding and changing is possible. Even deleting is allowed. – Antony Hatchkins Jun 12 '20 at 12:22

K Z · Answer 3 · 2012-09-27T15:52:08.567

Update:

Gareth Rees was right that point 1 & 3 were not valid in this case. Though I think point 2 and 4 are still valid, therefore I will leave theses here.

(I noticed that the request.POST object of both Pyramid(Pylon) and Django is some form of MultiDict. So perhaps it is a more common practice than making request.POST immutable.)

I can't speak for the Django guys, though it seems to me that it could because of some of these reasons:

Performence. immutable objects are "faster" over mutable ones in that they allow substantial optimizations. An object is immutable means that we can allocate space for it at creation time, and the space requirements are not changing. It also has things like copy efficiency and comparison efficiency because of it. Edit: this is not the case for QueryDict as Gareth Rees pointed out.
In the case of request.POST, it seems no activity in the server side should need to alter the request's data. And hence immutable objects are more suited, not to mention they have substantial performence advantage.
~~Immutable objects can be used as dict keys, which I suppose could be very useful somewhere in Django..~~ Edit: my mistake, immutable does not directly imply hashable; hashable objects however, are typically immutable as well.
When you pass around request.POST (especially to third-party plugins and out), you can expect that this request object from the user will remain unchanged.

In some way these reasons are also generic answers to "immutable vs mutable?" question. I am certain there are much more design considerations than above in the Django case.

The last case is really important. It's really about security. This is why Django provides `sessions` which is short-live way of getting and modifying data between states. — CppLearner, Sep 27 '12 at 04:02
Your point (1) can't be the answer in this case, because `POST` is a [`QueryDict` object](https://github.com/django/django/blob/master/django/http/__init__.py#L371), and these objects get no performance benefit from being immutable. And your point (3) can't be the answer, because `QueryDict` objects aren't hashable, and so can't used as dictionary keys. — Gareth Rees, Sep 27 '12 at 10:20
@GarethRees Thanks for pointing these out. Indeed I was wrong. I updated my answer to correct these. I should have paid more attention to `QueryDict` before I replied. — K Z, Sep 27 '12 at 15:53
@CppLearner The security point seems moot, e.g. `requests.POST._mutable = True; requests.POST['foo'] = 'bar'; request.POST._mutable = False` — Dan Passaro, Jan 07 '14 at 17:20

score 4 · Answer 4 · answered Mar 17 '15 at 15:24

4

I like it being immutable by default. As pointed out you can make it mutable if you need to but you must be explicit about it. It is like 'I know that I can make my form debugging a nightmare but I know what I am doing now.'

answered Mar 17 '15 at 15:24

pawel lukaszewicz

555
4
6

score 2 · Answer 5 · edited May 23 '17 at 12:26

2

I found this in a comment on Stack Answer https://stackoverflow.com/a/2339963

And it must be immutable so that it can be built lazily. The copy forces getting all the POST data. Until the copy, it may not all be fetched. Further, for a multi-threaded WSGI server to work reasonably well, it's helpful if this is immutable

edited May 23 '17 at 12:26

Community

1
1

answered Apr 30 '14 at 22:40

Seaux

3,459
2
28
27

score 1 · Answer 6 · answered Apr 12 '18 at 10:40

1

Please note: multipart request are immutable since Django 1.11 https://github.com/django/django/blob/stable/1.11.x/django/http/multipartparser.py#L292

They were mutable in previous versions.

answered Apr 12 '18 at 10:40

Alex Paramonov

2,630
2
23
27

Also mentioned in Django 1.11 [release notes](https://docs.djangoproject.com/en/3.0/releases/1.11/#miscellaneous). – djvg Feb 18 '20 at 19:40

django - why is the request.POST object immutable?

6 Answers6

Linked