uploading files with urllib to a FileField model in Django

Question

I have a django file model that has a models.FileField field and a form that is used to upload files to the server:

class UploadFile(model.Model):
    filename = models.FileField(upload_to='uploads')
    description = models.CharField(max_length=38, blank=True)

class UploadFileForm(ModelForm):
    class Meta:
        model = UploadFile
        fields = ('filename', 'description')

This is how the view function looks like:

def upload_file(request):
    if request.method == 'POST':
        form = UploadFileForm(request.POST, request.FILES)
        if form.is_valid():
            new_file = form.save()

Now I would like to have a python script that uses only the standard library to upload files to the UploadFile model using the view function above. However this code does not work because the POST request does not have a FILES method.

import urllib, urllib2
data = urllib.urlencode({'filename': open(uploadfile, "rb"),
                        'description': 'upload test'})
post_req = urllib2.Request(upload_file_url, data)
result = urllib2.urlopen(post_req)

How could I add a FILES method to the POST request to upload files using a python script? I have also tried to write a different view function that does not used the form but it is not working either.

I will use scp to copy the files over until I can get some help with this issue — gypaetus, Jun 06 '13 at 13:09

jfs · Accepted Answer · 2022-01-16T08:52:39.330

6

To see whether your django view works, you could test it using a browser:

<FORM action="{{ upload_file_url }}"
      enctype="multipart/form-data"
      method="POST">
  Description: <INPUT type="text" name="description" value="upload test"><BR>
  File to upload: <INPUT type="file" name="filename"><BR>
  <INPUT type="submit" value="Send">
</FORM>

It is complicated to replicate it programmatically using only stdlib.

To upload files as multipart/form-data you could use requests library:

import requests

response = requests.post(upload_file_url,
                         files={'filename': open(uploadfile,'rb'),
                                'description': 'upload test'})
print response.content

Or urllib2 and poster libraries:

import urllib2

import poster.encode
import poster.streaminghttp

opener = poster.streaminghttp.register_openers()

params = {'filename': open(uploadfile,'rb'), 'description': 'upload test'}
datagen, headers = poster.encode.multipart_encode(params)
response = opener.open(urllib2.Request(upload_file_url, datagen, headers))
print response.read()

edited Jan 16 '22 at 08:52

answered Jun 06 '13 at 18:12

jfs

399,953
195
994
1,670

Thanks, I may have to use the requests library in the future. At the moment I am using urllib.urlencode to encode postscript files since it is all I need but it does not work for binary files... – gypaetus Jun 06 '13 at 20:18
@MigueldeVal-Borro: urlencode() creates x-www-form-urlencoded data, but django's FileField expects multipart/form-data. It should *not* work for any file. Does your view work with the html form I've posted? – jfs Jun 06 '13 at 20:35
@JF Sebastian yes I had a similar form that worked. I also need to do the same thing with stdlib so I send something like `urllib.urlencode({"filename": "plot.ps", "data": f.read(), "description": "test upload"})` and define a view function that stores the data in the model and the file on disk. – gypaetus Jun 06 '13 at 20:54
1

@MigueldeVal-Borro: if you can't install any dependency then just add poster's source to your package (it is [pure Python](https://bitbucket.org/chrisatlee/poster/src/97d3c6fcb87785d1838549301f0092dec4229abf/poster?at=default) and doesn't require additional dependencies). – jfs Jun 06 '13 at 21:09
Don't forget to close the `uploadfile` file? – Vladius Mar 18 '15 at 22:13
The "complicated" reference is dead. If anyone who has interested by this way. That link https://github.com/pypa/setuptools/blob/main/setuptools/command/upload_docs.py is still alive. – Vanya Usalko Jan 13 '22 at 20:18
@Ivan Usalko: just use `requests` to upload (and/or edit the link in the answer) – jfs Jan 14 '22 at 17:26
@jfs Thanks a lot, it's certainly the best advice. But in my current case, I am working on a library that shouldn't use any external dependencies. And your's mentioned above reference helped me. – Vanya Usalko Jan 15 '22 at 19:56

score 0 · Answer 2 · edited May 23 '17 at 11:57

It ain't that easy, is not that request doesn't have a FILES method (which is not a method, is a dictionary). The problem is that Django doesn't recognizes files if the request doesn't have the Content-Type:multipart/form-data.

Note that FILES will only contain data if the request method was POST and the that posted to the request had enctype="multipart/form-data". Otherwise, FILES will be a blank dictionary-like object.

Here you can see how a request like that looks like: What should a Multipart HTTP request with multiple files look like?

https://docs.djangoproject.com/en/dev/ref/request-response/

So basically what you'll have to do is add that field to the request you're building and send it to django later.

You can take a look at this pages from django's doc to get more info about that and it will be great if you can sniff or see how the request is before sending it to django so you can know what is missing.

https://docs.djangoproject.com/en/dev/ref/forms/api/

Since there's not a easy/direct answer to this, I hope this can put you in the right direction.

Good luck!

I guess that I am missing how to add FILES to the request created with urllib2 — gypaetus, Jun 05 '13 at 23:12
Ok, you wont have to add `FILES` dictionary to the request. You'll have to conform the request properly and when Django parse it, it will populate the `request.FILES` dictionary by itself. — Paulo Bu, Jun 05 '13 at 23:28

uploading files with urllib to a FileField model in Django

2 Answers2