Django form failing validation and .save(commit=False) crashes

Question

I am trying to use ModelForms, but I really seem to be making a meal of it. The models are various subclasses from 'Answer.'

class Answer(models.Model):
question = models.ForeignKey(Question, on_delete=models.CASCADE)
user = models.ForeignKey(User, on_delete=models.CASCADE, default=1)
    class Meta:
        abstract = True
        ordering = ['user']

class Brainstorm(Answer):   
brain_bolt = models.CharField(max_length=200, default="")
responds_to = models.ForeignKey('self', models.SET_NULL, blank=True, null=True)
    class Meta:
        ordering = ['-pk', ]

The ModelForms all follow this pattern:

class BrainstormForm(ModelForm):
class Meta:
    model = Brainstorm
    fields = '__all__'

There are three possible patterns for Answers defined in the Question model:

MULTIPLE_ENTRY_OPTIONS = (
    ('S', 'Single Entry'),  # single pk. User changes are final.
    ('T', 'Change-Tracked Single Entry'),  # multiple-pks, but only the most recent is presented to the user
    ('M', 'Many answer instances are valid'),  # question requires many answers - suitable for brainstorming
)

A page may have multiple questions of different answer types and hence different forms, so rather than use a formset, I differentiate them individually with a prefix string of the question primary key and the answer primary key, which can then be unpacked again to get the question and Answer objects.

I have two function-based views for each page: page_view (responds to get) and answer (responds to POST). Page_view creates and fills the new form to present to the user. answer is supposed to respond to a POST request by saving the returned data. It might save it as a new entry or save it as an amendment.

def answer(request, project_id, user_id, next_question_page):
"""
answer handles getting submitted answers from http request objects into the
database, using either the 'answer-value' path (without Django Forms) or the "q-"
path, which uses form instantiator to unpack the form.
"""
# attempt to standardise saving of answers (votes, shorts, longs, E3, vE4 etc)
user = User.objects.get(username=user_id)
next_page = 'not_set'
for key, value in request.POST.items():
    if key.startswith("q"):
        q_number = re.search(r'q(\d+)#(\d+).+', key)
        pk_q = int(q_number.group(1))      # the digits are the question primary key
        pk_ans = int(q_number.group(2))  # these digits are the pk of the answer
        prefix = prefix = "q" + str(pk_q) + '#' + str(pk_ans)
        question = Question.objects.get(pk=pk_q)
        answer_class = ANSWER_CLASS_DICT[question.answer_type]
        model_instance = answer_class.objects.get(pk=pk_ans)
        form_instance = form_instantiator(question, request, instance=model_instance, prefix=prefix)
        print(form_instance)
        print(form_instance.fields('question'))
        if form_instance.is_valid:
            form_instance.save()
            if question.answer_type == 'BS':
                return HttpResponseRedirect(reverse('polls:project:page', args=(
                    project_id,
                    user_id,
                    question.man_page)))

forms.form_instantiator()

elif request and instance:

    form = FORM_CLASSES[question.answer_type](request.POST, prefix=prefix)
    form.fields['user'] = user
    form.fields['question'] = question
    temp_answer = form.save(commit=False)
    temp_answer.question = question
    temp_answer.user = user
    print('temp_answer:', temp_answer.question, temp_answer.user, temp_answer.brain_bolt)

else:
    form = FORM_CLASSES[question.answer_type]()
return form

Error is "form.save(commit=False) failed because the form didn't validate." I'm so confused, because after reading this (docs) I believed the commit=False would allow me to create an incomplete Answer object which I could further populate and then save.

I apologise for this enormous question; happy to take 'you can't get there from here' answers.

Request Method: POST Request URL: http://127.0.0.1:8000/polls/p1/cruicksa/pg11/ans

Django Version: 1.11.4 Python Version: 3.6.0 Installed Applications: ['polls.apps.PollsConfig', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'import_export'] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "C:\Users\cruicksa\AppData\Local\Continuum\Anaconda3\lib\site-packages\django\core\handlers\exception.py" in inner 41. response = get_response(request)

File "C:\Users\cruicksa\AppData\Local\Continuum\Anaconda3\lib\site-packages\django\core\handlers\base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request)

File "C:\Users\cruicksa\AppData\Local\Continuum\Anaconda3\lib\site-packages\django\core\handlers\base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "C:\Users\cruicksa\AppData\Local\Continuum\Anaconda3\lib\site-packages\django\contrib\auth\decorators.py" in _wrapped_view 23. return view_func(request, *args, **kwargs)

File "H:\Workspace\Django_Code\Acejet_development\polls\views.py" in answer 164. form_instance = form_instantiator(question, request, instance=model_instance, prefix=prefix)

File "H:\Workspace\Django_Code\Acejet_development\polls\forms.py" in form_instantiator 191. temp_answer = form.save(commit=False)

File "C:\Users\cruicksa\AppData\Local\Continuum\Anaconda3\lib\site-packages\django\forms\models.py" in save 458. 'created' if self.instance._state.adding else 'changed',

Exception Type: ValueError at /polls/p1/cruicksa/pg11/ans Exception Value: The Brainstorm could not be created because the data didn't validate.

Answer 1

is_valid is callable so it should be

if form_instance.is_valid():

You missed ().

Answer 2

The problem with posting a long question is that it makes it difficult to see where the problems started.

I was using modelforms, but several of fields should have been excluded - user, for example, because: a. the user should not be able to answer on behalf of others, and b. the app should not be sending the full list of users to each user with every page load.

My design problem was receiving user input for other fields, and adding back the 'question' and 'user' data to the form before calling validation. I also had a fair bit of code that worked but needed to change so I could use 'forms' properly (I was creating answers, loading them in forms, sending them to users, then hoping to match them up again sometimes and not other times).

Working from this answer, I now have my app working again and using forms correctly.

I migrated the views for 'page' and 'answer' to get and post of a class-based view, and I replaced form_instantiator with form_getter and form_post functions.

The thing I had to learn was how to populate a data dictionary to use in creating a form.

    @method_decorator(login_required)
def post(self, request, project_id, page_num):
    self.user = User.objects.get(username=request.user.username)
    self.next_page = 'not_set'
    self.form_items = {}
    prefixes = []
    for key, value in request.POST.items():
        if key.startswith("q"):
            q_number = re.search(r'q(\d+)#', key)
            pk_q = int(q_number.group(1))  # the digits are the question primary key
            prefix = 'q' + str(pk_q) + '#'
            if prefix not in prefixes:
                prefixes.append(prefix)
                self.form_items[prefix] = {key: value for key, value in request.POST.items() if prefix in key}
                #   https: // stackoverflow.com / questions / 45441048 / how - do - i - filter - a - dictionary - based - on - the - partial - string - matches
                question = Question.objects.get(pk=pk_q)
                self.form_items[prefix].update({(prefix + '-question'): question.pk,
                                                (prefix +'-user'): self.user.pk})
                form = form_post(value=self.form_items[prefix], question=question, user=self.user)
                form.save()

This still feels a bit hacky, depending as it does on adding the prefix to a hard-coded version of the data field name ('-question'). This is necessary since I'm wrapping several forms that might or might not be of the same type in a single 'submit'. Also, flow goes through request.POST in two loops, (one is a single line comprehension) which is wasteful.

But with question and user added back, a valid form can now be created and saved. My thanks to everyone who tried to help.

Where I had form.fields['question'] = question, I should have been editing the request.POST input into a proper complete set of data.