9

I need some examples with file operations using in memory and temporary files.

I searched a lot for a good example /tutorial and I found just basic read/write/append operations.

I need to understand how can I read in Django a file that is uploaded(an image) before the save(post) is finished.

Because Django is Python, I think is better to understand first in python.

I checked the Django documentation/examples about, but it is not very clear so I need to understand first in Python then in Django how the operations are working, not just copy and paste.

I know how to use ImageFields, upload default operation, I'm interested only in using "in memory and temporary files."

I want to use this in combination with a crop function. So the user can upload 1, 2, 3... images, and with a javascript crop script I get the coordinates in a hidden field. After javascript simulated crop I show the user a thumbnail with the crop, how will look like ratio

The user can change his mind and can edit/update or delete a file before saving.

Now depending on the file size it can be keep in memory or write.

Lance Goyke
  • 53
  • 7
user3541631
  • 3,686
  • 8
  • 48
  • 115
  • i feel you want to know how to upload an image to a django server. is that correct ? – Vasif Aug 24 '17 at 16:05
  • I know how to upload a Image, using ImageField etc, I need something specific regarding in memory and temporary files – user3541631 Aug 24 '17 at 16:06
  • i tried to write down the best i can get out of your question. Feel free to add what you are looking for as a comment there.. – Vasif Aug 24 '17 at 16:13
  • It would help if you said a bit more about what exactly you want to do with the in-memory/temp file. Everything in Django is build on top of native python IO library. https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files – Jarek.D Aug 24 '17 at 16:28
  • OK now I see what you're trying to achieve. In this case you don't really need to send the file to the server. Crop and preview on the client and save to server only when user accepts the crop – Jarek.D Aug 24 '17 at 16:32
  • https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem – Vasif Aug 24 '17 at 16:34
  • The crop will be done with Pillow(multiple types of crops) behind. I don't do on the front-end because of multiple crops and I need to make it work on IE9. – user3541631 Aug 24 '17 at 16:34
  • If you want to use Pillow you have to upload the file as normal, read it from the request as per suggestion from my answer below, create thumbnails, save the thumbnails and show them back to the user. – Jarek.D Aug 24 '17 at 16:45

4 Answers4

19

When a file is uploaded Django will do one of two things: store it in memory if the file is small (< 2 MB last time I checked), or store it as a temporary file on disk if it's large. This behavior is configurable via the FILE_UPLOAD_HANDLERS setting. So, your web server and Django take care of the actual upload and storage, but it's your job to process the file before the request is over, otherwise the file is deleted.

Uploaded files are accessible through the request.FILES property. Each key in FILES will match the name of the file input on your <form>. The value is an UploadedFile object, which is a stream you can use to read the file data.

For example, say you have an <input name="img" type="file" /> and you want to detect if the image is completely white. You don't need to store the file for this, you just need to load it into memory, process it to get the result and then let it be discarded. 

from PIL import Image

def some_view(request):
    if request.method == 'POST':
        img_file = request.FILES['img']

        if img_file.size > 2000000:
            return HttpResponseBadRequest()

        img = Image.open(img_file)

        # analyze the image...

Another possibility is that someone is uploading a backup file that is quite large (lets say 2 GB), and you need to store it somewhere. It's effectively the same thing, except we read the file into memory in chunks, then write each chunk to disk somewhere else so that it's saved after the request finishes. 

def some_view(request):
    if request.method == 'POST':
        backup_file = request.FILES['backup_file']
        with open('some/file/name.bak', 'wb+') as destination:
            for chunk in backup_file.chunks():
                destination.write(chunk)

        # file is saved

When the request is over, the uploaded file is stored at some/file/name.bak.

Whether it's in memory or a temporary file is usually not important because the interface is the same. You can read a temporary file just like you can read an in memory file.

mindcruzer
  • 834
  • 5
  • 9
3

You can set which upload handler is used for a Django form: https://docs.djangoproject.com/en/1.11/ref/files/uploads/#module-django.core.files.uploadhandler

There are two built in options:

The way Django seems to work is by walking through the list of all upload handlers: https://docs.djangoproject.com/en/1.11/topics/http/file-uploads/#upload-handlers

For each upload handler it will check for a specific condition and if that condition is true, it will activate and use that upload handler.

The InMemoryUploadHandler, for example, is activated and used when files are below a certain size. The TemporaryFileUploadHandler is used when files are very large.

You will be able to access the data and files from the request object.

2

If you're looking at Python specific (not Django), then it sounds like you may be interested in the tempfile module:

https://docs.python.org/3/library/tempfile.html

Adapting the example from the document above:

import tempfile

fp = tempfile.TemporaryFile()
fp.write(b'Hello world!')
# Closing automatically deletes the tempfile
fp.close()

You can see this works fairly similarly to generic read/write file operations.

Proto
  • 764
  • 1
  • 7
  • 22
1

If I understand correctly you are looking for a way to access the uploaded file before it is saved using instance.save() or similar.

If that is the case, you might try to read the file directly from request:

if my_form.is_valid():
    data = request.FILES['myfile'].read()
Jarek.D
  • 1,274
  • 1
  • 8
  • 18