django: Is there anyway to serve a file directly from memory

Question

So I have users requesting their files from a django server inside which some processes take place on the requested file. Currently I am writing the file to the disk and serve it from there.

Is there anyway to serve the file directly to the user from the memory for a better efficiency and performance (using x-sendfile)?

Here is my current view:

def ServeSegment(request, segmentID):
    segments =[]
    obj = MainFile.objects.filter(owner=request.user)
    file_name = MainFile.objects.get(file_id=segmentID).file_name
    file_suffix = file_name = MainFile.objects.get(file_id=segmentID).file_suffix
    if request.method == 'GET':
        hosts = settings.HOSTS
        for i in hosts:
            try:
                url = 'http://' + i + ':8000/foo/'+str(segmentID)
                r = requests.get(url, timeout=1, stream=True)
                if r.status_code == 200:
                    segments.append(r.content)
            except:
                continue
        instance = SeIDA(filename='test', x=settings.M, y=settings.N)
        docfile = instance.decoder(segments)
        with open('/tmp/Files/'+file_name, 'wb') as f:
            f.write(docfile)
            response = HttpResponse()
            response['Content-Disposition'] = 'attachment; filename={0}'.format(file_name)
            response['X-Sendfile'] = "/tmp/Files/{0}".format(file_name)
            return response

Note: SeIDA module being used, encodes data into N segments such that M segments is sufficed to construct the data. So in the view above I am retrieving the segments from the storage servers and combining them.

My question being: How can I directly serve the docfile without saving it. Is there anyway?

I think the answer can be found here: http://stackoverflow.com/a/909088/2372812 — Tom Dalton, Jan 24 '16 at 10:54

score 1 · Accepted Answer · answered Jan 24 '16 at 11:50

You can't serve it from memory using X-Sendfile, because Apache is running in a different process it even on a completely separate machine.

However in this case using sendfile isn't really making things more efficient at all. Since as you say the file is already in memory, you should serve it straight from there without going via Apache at all: just return it in the response.

docfile = instance.decoder(segments)
response = HttpResponse()
response.write(docfile)
response['Content-Disposition'] = 'attachment; filename={0}'.format(file_name)
return response

Didn't know I could directly write to response, thanks for teaching me that. Is this approach proper for handling numerous concurrent connections requesting files? — Amir Afianian, Jan 24 '16 at 12:15

django: Is there anyway to serve a file directly from memory

1 Answers1