8

I have a multi-file upload and want to limit users to 3 uploads each. My problem is that I need to know how many files a user has already created in the DB and how many they are currently uploading (they can upload multiple files at once, and can upload multiple times).

I have attempted many things, including:

Creating a validator (the validator was passed the actual file being added, not a model, so I couldn't access the model to get it's id to call if StudentUploadedFile.objects.filter(student_lesson_data=data.id).count() >= 4:).

Doing the validation in clean(self): (clean was only passed one instance at a time and the DB isn't updated till all files are cleaned, so I could count the files already in the DB but couldn't count how many were currently being uploaded).

Using a pre-save method (If the DB was updated between each file being passed to my pre-save method it would work, but the DB is only updated after all the files being uploaded have passed through my pre-save method).

My post-save attempt:

@receiver(pre_save, sender=StudentUploadedFile)
def upload_file_pre_save(sender, instance, **kwargs):

    if StudentUploadedFile.objects.filter(student_lesson_data=instance.data.id).count() >= 4:
        raise ValidationError('Sorry, you cannot upload more than three files')

edit:

models.py

class StudentUploadedFile(models.Model):
    student_lesson_data = models.ForeignKey(StudentLessonData, related_name='student_uploaded_file', on_delete=models.CASCADE)
    student_file = models.FileField(upload_to='module_student_files/', default=None)

views.py

class StudentUploadView(View):
    def get(self, request):
        files_list = StudentUploadedFile.objects.all()
        return render(self.request, 'users/modules.html', {'student_files': files_list})

    def post(self, request, *args, **kwargs):
        form = StudentUploadedFileForm(self.request.POST, self.request.FILES)
        form.instance.student_lesson_data_id = self.request.POST['student_lesson_data_id']

        if form.is_valid():
            uploaded_file = form.save()

            # pass uploaded_file data and username so new file can be added to students file list using ajax
            # lesson_id is used to output newly added file to corresponding newly_added_files div
            data = {'is_valid': True, 'username': request.user.username, 'file_id': uploaded_file.id, 'file_name': uploaded_file.filename(),
            'lesson_id': uploaded_file.student_lesson_data_id, 'file_path': str(uploaded_file.student_file)}
        else:
            data = {'is_valid': False}
        return JsonResponse(data)

template.py

<form id='student_uploaded_file{{ item.instance.id }}'>
                                                {% csrf_token %}
                                                <a href="{% url 'download_student_uploaded_file' username=request.user.username file_path=item.instance.student_file %}" target='_blank'>{{ item.instance.filename }}</a>
                                                <a href="{% url 'delete_student_uploaded_file' username=request.user.username file_id=item.instance.id %}" class='delete' id='{{ item.instance.id }}'>Delete</a>
                                            </form>

js

$(function () {
    // open file explorer window
    $(".js-upload-photos").on('click', function(){
        // concatenates the id from the button pressed onto the end of fileupload class to call correct input element
        $("#fileupload" + this.id).click();
     });

    $('.fileupload_input').each(function() {
        $(this).fileupload({
            dataType: 'json',
            done: function(e, data) { // process response from server
            // add newly added files to students uploaded files list
            if (data.result.is_valid) {
                $("#newly_added_files" + data.result.lesson_id).prepend("<form id='student_uploaded_file" + data.result.file_id +
                "'><a href='/student_hub/" + data.result.username + "/download_student_uploaded_file/" +
                data.result.file_path + "' target='_blank'>" + data.result.file_name + "</a><a href='/student_hub/" + data.result.username +
                "/delete_student_uploaded_file/" + data.result.file_id + "/'  class='delete' id=" + data.result.file_id + ">Delete</a></form>")
            }
            }
        });
    });

UPDATE: forms.py

class StudentUploadedFileForm(forms.ModelForm):
    student_file = forms.FileField(widget=forms.ClearableFileInput(attrs={'multiple': True}))

view.py

class StudentUploadView(View):
    model = StudentUploadedFile
    max_files_per_lesson = 3

    def post(self, request, *args, **kwargs):
        lesson_data_id = request.POST['student_lesson_data_id']
        current_files_count = self.model.objects.filter(
            student_lesson_data_id=lesson_data_id
        ).count()
        avail = self.max_files_per_lesson - current_files_count
        file_list = request.FILES.getlist('student_file')
        print(len(file_list))
        if avail - len(file_list) < 0:
            return JsonResponse(data={
                'is_valid': False,
                'reason': f'Too many files: you can only upload {avail}.'
            })
        else:
            for f in file_list:
                print(f)
                
        data = {'test': True}
        return JsonResponse(data)

Thank you.

horse
  • 479
  • 7
  • 25
  • why don't you count them by the user_id ? count uploaded data filtered by user – A Khalili Jul 13 '20 at 07:47
  • Your `upload_file_pre_save` function refers to an `UploadedFile` model filtered on a `data_id` field. Your models.py declares a `StudentUploadedFile` model with a `student_lesson_data` field. Are these the same thing? It's also unclear how `StudentUploadedFile` is linked to your user. Is `StudentLessonData` a custom user? – MattRowbum Jul 14 '20 at 02:22
  • Sorry, I forgot that I simplified the code I originally posted to try to make it easier to understand. I fixed those things up, yes they're the same. For each lesson a ```Student``` is enrolled in they have a ```StudentLessonData``` object which holds data specific to that ```Student```/lesson. ```StudentUploadedFile``` has a one-to-many relationship to ```StudentLessonData``` (one ```StudentLessonData``` can have many ```StudentUploadedFile```s). – horse Jul 14 '20 at 02:34
  • Your question states that you want to 'limit users to 3 uploads each'. Is that 3 uploads per `Student`, or 3 uploads per `StudentLessonData`? – MattRowbum Jul 14 '20 at 03:45
  • Where did `StudentModuleData` come from? You haven't mentioned that model before. – MattRowbum Jul 14 '20 at 04:03
  • Ahh sorry, I am getting everything mixed up - I'll pay close attention to anything else I post to make sure everything makes sense. I meant that a single ```StudentLessonData``` can have a max of 3 ```StudentUploadedFile``` attached to it. Therefore one ```Student``` may have zero-or-more ```StudentLessonData``` and each of those up to 3 ```StudentUploadedFile```. – horse Jul 14 '20 at 04:09
  • Please go through the following link for reference: https://stackoverflow.com/questions/10105411/how-to-limit-the-maximum-files-chosen-when-using-multiple-file-input – Leeds Leeds Jul 19 '20 at 17:46

4 Answers4

1

I've tried using a PyPi package and it works flawlessly. I'm gonna go out on a limb here and assume that you are open to editing the package code to fix any errors you encounter due to compatibility issues since most of the packages that haven't been updated in quite a while might face them.

To solve the problem of limiting the number of files a user can upload, django-multiuploader package would be of enormous help and would honestly do more than you ask for. And yes, it uses JQuery form for uploading multiple files.

How to use it?

Installation and pre-usage steps

Installation

pip install django-multiuploader
python3 manage.py syncdb
python3 manage.py migrate multiuploader

In your settings.py file :

MULTIUPLOADER_FILES_FOLDER = ‘multiuploader’ # - media location where to store files

MULTIUPLOADER_FILE_EXPIRATION_TIME = 3600  # - time, when the file is expired (and it can be cleaned with clean_files command).

MULTIUPLOADER_FORMS_SETTINGS =

{
'default': {
    'FILE_TYPES' : ["txt","zip","jpg","jpeg","flv","png"],
    'CONTENT_TYPES' : [
            'image/jpeg',
            'image/png',
            'application/msword',
            'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
            'application/vnd.ms-excel',
            'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
            'application/vnd.ms-powerpoint',
            'application/vnd.openxmlformats-officedocument.presentationml.presentation',
            'application/vnd.oasis.opendocument.text',
            'application/vnd.oasis.opendocument.spreadsheet',
            'application/vnd.oasis.opendocument.presentation',
            'text/plain',
            'text/rtf',
                ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
},
'images':{
    'FILE_TYPES' : ['jpg', 'jpeg', 'png', 'gif', 'svg', 'bmp', 'tiff', 'ico' ],
    'CONTENT_TYPES' : [
        'image/gif',
        'image/jpeg',
        'image/pjpeg',
        'image/png',
        'image/svg+xml',
        'image/tiff',
        'image/vnd.microsoft.icon',
        'image/vnd.wap.wbmp',
        ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
},
'video':{
    'FILE_TYPES' : ['flv', 'mpg', 'mpeg', 'mp4' ,'avi', 'mkv', 'ogg', 'wmv', 'mov', 'webm' ],
    'CONTENT_TYPES' : [
        'video/mpeg',
        'video/mp4',
        'video/ogg',
        'video/quicktime',
        'video/webm',
        'video/x-ms-wmv',
        'video/x-flv',
        ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
},
'audio':{
    'FILE_TYPES' : ['mp3', 'mp4', 'ogg', 'wma', 'wax', 'wav', 'webm' ],
    'CONTENT_TYPES' : [
        'audio/basic',
        'audio/L24',
        'audio/mp4',
        'audio/mpeg',
        'audio/ogg',
        'audio/vorbis',
        'audio/x-ms-wma',
        'audio/x-ms-wax',
        'audio/vnd.rn-realaudio',
        'audio/vnd.wave',
        'audio/webm'
        ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
}}

Take note of that MAX_FILE_NUMBER, right within their lies the answer to your question. Have a look at the source once you install this and try implementing it on your own if you want. It might be fun.

Refer for further instructions : django-multiuploader package on pypi

Roast Biter
  • 651
  • 1
  • 6
  • 22
1

I guess that you can use multi file upload in Django hasn't trickled to the community yet. Excerpt:

If you want to upload multiple files using one form field, set the multiple HTML attribute of field’s widget:

# forms.py

from django import forms

class FileFieldForm(forms.Form):
    file_field = forms.FileField(widget=forms.ClearableFileInput(attrs={'multiple': True}))

Your form and view structure is also very contrived, excluding fields from a form and then setting values injected via HTML form on the model instance. However, with the code shown, the model instance would never exist as the form has no pk field. Anyway - to focus on the problem that needs fixing...

In the form, request.FILES is now an array:

class StudentUploadView(View):
    model = StudentUploadedFile
    max_files_per_lesson = 3
    def post(request, *args, **kwargs):
        lesson_data_id = request.POST['student_lesson_data_id']
        current_files_count = self.model.objects.filter(
            student_lesson_data_id=lesson_data_id
        ).count()
        avail = self.max_files_per_lesson - current_files_count
        file_list = request.FILES.get_list('student_file')
        if avail - len(file_list) < 0:
            return JsonResponse(data={
                'is_valid': False,
                'reason': f'Too many files: you can only upload {avail}.'
            })
        else:
            # create one new instance of self.model for each file
            ...

Addressing comments: From an aesthetic perspective, you can do a lot with styling...

However, uploading async (separate POST requests), complicates validation and user experience a lot:

  • The first file can finish after the 2nd, so which are you going to deny if the count is >3.
  • Frontend validation is hackable, so you can't rely on it, but backend validation is partitioned into several requests, which from the user's point of view is one action.
  • But with files arriving out of order, some succeeding and some failing, how are you going to provide feedback to the user?
  • If 1, 3 and 4 arrive, but user cares more about 1, 2, 3 - user has to take several actions to correct the situation.

One post request:

  • There are no out of order arrivals
  • You can use a "everything fails or everything succeeds" approach, which is transparent to the end user and easy to correct.
  • It's likely that file array order is user preferred order, so even if you allow partial success, you're likely to do the right thing.
  • Thank you. I tried your suggestion (I updated the code above), yet when I ```print(len(file_list))``` prior to ```if avail - len(file_list) < 0:``` it prints ```1``` and repeats that for the number of files which are being uploaded – horse Jul 19 '20 at 10:50
  • Did you ditch JQuery? You should be sending *one* POST request, not one for each file. –  Jul 19 '20 at 12:20
  • Oh ok. I was hoping to keep the jquery as i was wanting to only have the one button (rather than a 'select file' button and an 'upload' button, the jquery just has a single 'choose file' button and automatically uploads the files without a second button press). And it also adds the file to the user's uploaded files list without a page refresh. But if its complicating things il drop the jquery. Thank you – horse Jul 19 '20 at 22:37
0

So the gist of it is that you are using jQuery .each to upload images via AJAX. Each POST request to your Django view is a single file upload, but there might be multiple requests at the same time.

Try this:

forms.py:

class StudentUploadedFileForm(forms.ModelForm):

    class Meta:
        model = StudentUploadedFile
        fields = ('student_file', )

    def __init__(self, *args, **kwargs):
        """Accept a 'student_lesson_data' parameter."""
        self._student_lesson_data = kwargs.pop('student_lesson_data', None)
        super(StudentUploadedFileForm, self).__init__(*args, **kwargs)

    def clean(self):
        """
        Ensure that the total number of student_uploaded_file instances that
        are linked to the student_lesson_data parameter are within limits."""
        cleaned_data = super().clean()
        filecount = self._student_lesson_data.student_uploaded_file.count()
        if filecount >= 3:
            raise forms.ValidationError("Sorry, you cannot upload more than three files")
        return cleaned_data

views.py:

class StudentUploadView(View):
    def get(self, request):
        # stuff ...

    def post(self, request, *args, **kwargs):
        sld_id = request.POST.get('student_lesson_data_id', None)
        student_lesson_data = StudentLessonData.objects.get(id=sld_id)
        form = StudentUploadedFileForm(
            request.POST,
            request.FILES,
            student_lesson_data=student_lesson_data
        )

        if form.is_valid():
            uploaded_file = form.save()
            # other stuff ...
MattRowbum
  • 2,162
  • 1
  • 15
  • 20
  • Thank you very much for your response, however this hasn't quite got it. I put ```print``` statements inside the ```__init__``` and ```clean``` functions and saw that it executes ```__init__``` followed by ```clean``` for each file added in the multifile upload (if I ctrl-click 3 files, it executes 3 times). However if I ```print(filecount)``` before the ```if``` statement in the ```clean``` method it prints the number of files in the DB prior to the current files that are uploaded, and doesn't increment with each file. – horse Jul 14 '20 at 08:52
  • No problem. Based on your testing, I think your best option will be to update your javascript so that all images are submitted in one POST. – MattRowbum Jul 14 '20 at 12:34
  • Ok no worries. I'll have a look at the javascript a bit later when I have time. If I get it submitting in one POST I'm guessing your solution would do the trick. Thank you – horse Jul 14 '20 at 23:24
0

You can use the below setting to set the number of files that can be uploaded. (New in Django 3.2.18. )

DATA_UPLOAD_MAX_NUMBER_FILES

The maximum number of files that may be received via POST in a multipart/form-data encoded request before a SuspiciousOperation (TooManyFiles) is raised. You can set this to None to disable the check. Applications that are expected to receive an unusually large number of file fields should tune this setting.

The number of accepted files is correlated to the amount of time and memory needed to process the request. Large requests could be used as a denial-of-service attack vector if left unchecked. Since web servers don’t typically perform deep request inspection, it’s not possible to perform a similar check at that level.

The default value is 100.

SuperNova
  • 25,512
  • 7
  • 93
  • 64