How to upload a file from request.FILES using Celery in Django?

Question

There's a model called Post which has an image field. The image field gets its value from a form. I don't want media files to be stored locally. So I'm using S3Boto3Storage as my file storage. Is there a way to save the Post model object using celery (so that the image file gets uploaded by celery to not keep the user waiting). I tried writing a celery task to save the object. But apparently you can't just send a file to Celery as a parameter. This is what I have so far:

views.py:

tasks.upload_image_task.delay(
    user_pk,
    post_body,
    request.FILES.get("image"),
)

tasks.py:

@shared_task
def upload_image_task(user_pk, body, image):
    user = User.objects.get(pk=user_pk)
    post = Post.objects.create(user=user, body=body, image=image, post_type="IMG")
    post.save()

This will result in Object of type InMemoryUploadedFile is not JSON serializable. I also read that first you can save the file on local and then upload the file using a task. But I want to upload it directly.

score 1 · Accepted Answer · answered Jul 23 '23 at 13:13

Amazon S3 has a concept of presigned post url, which you can use to directly upload the photo from the user browser instead of uploading it from your django backend. That should make your API faster but you still have to wait while the file is uploaded on the browser.

But for other reasons if you still want to post image to the backend, you cannot be using celery unless you have common file storage accessible from both celery worker and django process. In that case, you can first save the file locally and send the path to celery task input arguments. But if you don't have such storage, you can run the uploading of file asynchronously after sending the response. You can find answers regarding that here.

How to upload a file from request.FILES using Celery in Django?

1 Answers1