django storages aws s3 delete file from model record

Question

I have based my django aws S3 solution on https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html.

Now I am trying to find a way to delete a row in the model that contains the S3 file. I am able to delete the row with .delete() but it doesnt delete in S3.

How do I make the delete row to also delete in S3 ?

Below are my code:

https://gitlab.com/firdausmah/railercom/blob/master/railercomapp/api.py

@api_view(['POST'])
def delete_employee(request):
    # ----- YAML below for Swagger -----
    """
    description: This API deletes employee
    parameters:
      - name: employee_id
        type: integer
        required: true
        location: form
    """
    employee_id = request.POST['employee_id']
    employee = Employee.objects.get(id = employee_id)
    logging.debug(f"API employee username {employee.username}")
    employee.delete() <---------- here is where the delete row happens
    return Response("Employee Deleted!", status=status.HTTP_200_OK)

https://gitlab.com/firdausmah/railercom/blob/master/railercomapp/models.py

class Employee(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE, related_name='employee')
    company = models.ForeignKey(Company)
    username = models.CharField(max_length=30, blank=False)
    upload = models.FileField(blank=True) <--- S3 field

https://gitlab.com/firdausmah/railercom/blob/master/railercom/settings.py (AWS settings)

AWS_ACCESS_KEY_ID = config('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = config('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET_NAME')
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}

DEFAULT_FILE_STORAGE = 'railercomapp.storage_backends.MediaStorage'

https://gitlab.com/firdausmah/railercom/blob/master/railercomapp/storage_backends.py from storages.backends.s3boto3 import S3Boto3Storage

class MediaStorage(S3Boto3Storage):
    location = 'media/yy'
    file_overwrite = False

Håken Lid · Accepted Answer · 2020-12-20T11:59:59.817

You have to explicitly delete the file. You could write a post delete signal or do it in the delete_employee function.

employee.upload.delete(save=False)  # delete file
employee.delete()  # delete model instance

The docs for FileField.delete() explains this.

Note that when a model is deleted, related files are not deleted. If you need to cleanup orphaned files, you’ll need to handle it yourself (for instance, with a custom management command that can be run manually or scheduled to run periodically via e.g. cron).

You should also make sure that there's no other FileField that references the exact same file before deleting it.

Thanks, you just saved my coding career – Shubham Gupta Dec 20 '20 at 11:29 — Shubham Gupta, Dec 20 '20 at 11:29

score 1 · Answer 2 · answered Jun 02 '22 at 16:33

For all of my broski who struggle to delete the folder with django-storages.

Let's consider a real case. I have a dynamic path that each file stored in the folder and I had to implement a cleanup.

def get_upload_path(instance, filename):
    return os.path.join(
        'organizations',
        'files',
        str(instance.organization.pk),
        str(instance.hash),
        str(filename)
    )

file = models.FileField(
    upload_to=get_upload_path
)

The problem with my case was that I could not delete a folder with django-storages during the cleanup. instance.file.name raises an error because you cannot have an absolute path to the file with django-storages. In order to delete a folder, you should use storage.delete() because since you cannot have an absolute path you cannot delete a folder with a straightforward manner (for example shutil.rmtree(...)).

My clean-up implementation is a little complex but it's solid. In my case, I used the pre_delete signal and advised you to do the same.


from django.core.files.storage import get_storage_class

default_storage = get_storage_class()()

@receiver(pre_delete, sender=OrganizationFile)
def delete_has_folder(sender, instance, *args, **kwargs):
    # get filename that will be equals to the relative path but not actually the filename
    path = Path(instance.file.name)
    # get a parent folder str
    folder_path = str(path.parent)
    
    # delete a file
    instance.file.delete()
    # delete a folder. 
    #  shutil.rmtree(absolute_path) wouldn't work
    #  because instance.file.path will raise an error.
    #  hence the only way is to delete with a storage default_storage.delete
    default_storage.delete(folder_path)

    logger.info(f'Pre delete {instance}. Deleted the hash folder {folder_path}')

score 0 · Answer 3 · answered Feb 13 '20 at 05:10

I used this:

import boto3

client = boto3.client('s3')
client.delete_object(Bucket='mybucketname', Key='myfile.whatever')

But I'm trying to find a way to do it with the ImageFile object of my model, or maybe with some configuration of my storage class:

from storages.backends.s3boto3 import S3Boto3Storage 

class MediaStorage(S3Boto3Storage):    
    location = 'media'    
    file_overwrite = True

django storages aws s3 delete file from model record

3 Answers3