Correct way to save nested formsets in Django

Question

I have a 3-level Test model I want to present as nested formsets. Each Test has multiple Results, and each Result can have multiple Lines. I am following Yergler's method for creating nested formsets, along with this SO question that updates Yergler's code for more recent Django version (I'm on 1.4)

I am running into trouble because I want to use FormSet's "extra" parameter to include an extra Line in the formset. The ForeignKey for each Line must point to the Result that the Line belongs to, but cannot be changed by the user, so I use a HiddenInput field to contain the Result in each of the FormSet's Lines.

This leads to "missing required field" validation errors because the result field is always filled out (in add_fields), but the text and severity may not (if the user chose not to enter another line). I do not know the correct way to handle this situation. I think that I don't need to include the initial result value in add_fields, and that there must be a better way that actually works.

Update below towards bottom of this question

I will gladly add more detail if necessary.

The code of my custom formset:

LineFormSet = modelformset_factory(
    Line,  
    form=LineForm,
    formset=BaseLineFormSet,
    extra=1)

class BaseResultFormSet(BaseInlineFormSet):

    def __init__(self, *args, **kwargs):
        super(BaseResultFormSet, self).__init__(*args, **kwargs)

    def is_valid(self):
        result = super(BaseResultFormSet, self).is_valid()

        for form in self.forms:
            if hasattr(form, 'nested'):
                for n in form.nested:
                    n.data = form.data
                    if form.is_bound:
                        n.is_bound = True  
                    for nform in n:
                        nform.data = form.data
                        if form.is_bound:
                            nform.is_bound = True
                    # make sure each nested formset is valid as well
                    result = result and n.is_valid()
        return result

    def save_all(self, commit=True):
        objects = self.save(commit=False)

        if commit:
            for o in objects:
                o.save()

        if not commit:
            self.save_m2m()

        for form in set(self.initial_forms + self.saved_forms):
            for nested in form.nested:
                nested.save(commit=commit)

    def add_fields(self, form, index):
        # Call super's first
        super(BaseResultFormSet, self).add_fields(form, index)

        try:
            instance = self.get_queryset()[index]
            pk_value = instance.pk
        except IndexError:
            instance=None
            pk_value = hash(form.prefix)


        q = Line.objects.filter(result=pk_value)
        form.nested = [
            LineFormSet(
                queryset = q, #data=self.data, instance = instance, prefix = 'LINES_%s' % pk_value)]
                prefix = 'lines-%s' % pk_value,
                initial = [
                    {'result': instance,}
                ]
            )]

Test Model

class Test(models.Model):
    id = models.AutoField(primary_key=True, blank=False, null=False)

    attempt = models.ForeignKey(Attempt, blank=False, null=False)
    alarm = models.ForeignKey(Alarm, blank=False, null=False)

    trigger = models.CharField(max_length=64)
    tested = models.BooleanField(blank=False, default=True)

Result Model

class Result(models.Model):
    id = models.AutoField(primary_key=True)   
    test = models.ForeignKey(Test)

    location = models.CharField(max_length=16, choices=locations)
    was_audible = models.CharField('Audible?', max_length=8, choices=audible, default=None, blank=True)

Line Model

class Line(models.Model):
    id = models.AutoField(primary_key=True)
    result = models.ForeignKey(Result, blank=False, null=False)

    text = models.CharField(max_length=64)
    severity = models.CharField(max_length=4, choices=severities, default=None)

---

Update

Last night I added this to my LineForm(ModelForm) class:

def save(self, commit=True):
    saved_instance = None

    if not(len(self.changed_data) == 1 and 'result' in self.changed_data):
            saved_instance = super(LineForm, self).save(commit=commit)

    return saved_instance

It ignores the requests to save if only the result (a HiddenInput) is filled out. I haven't run into any problems with this approach yet, but I haven't tried adding new forms.

Answer 1

When I used extra on formsets in similar situation I ended up having to include all the required fields from the model in the form, as HiddenInputs. A bit ugly but it worked, curious if anyone has a hack-around.

edit
I was confused when I wrote above, I'd just been working on formsets using extra with initial to pre-fill the extra forms and also I hadn't fully got all the details of your questions.

If I understand correctly, where you instantiate the LineFormSets in add_fields each of those will point to the same Result instance?

In this case you don't really want to supply result in initial due to the problems you're having. Instead you could remove that field from the LineForm model-form altogether and customise the LineFormSet class something like:

class LineFormSet(forms.BaseModelFormSet):
    # whatever other code you have in it already
    # ...
    # ...
    def __init__(self, result, *args, **kwargs):
        super(LineFormSet, self).__init__(*args, **kwargs)
        self.result = result

    def save_new(self, form, commit=True):
        instance = form.save(commit=False)
        instance.result = self.result
        if commit:
            instance.save()
        return instance

    def save_existing(self, form, instance, commit=True):
        return self.save_new(form, commit)

(this should be ok in Django 1.3 and 1.4, not sure other versions)

so the relevant part of your add_fields method would look like:

   form.nested = [
        LineFormSet(
            result = instance,
            queryset = q, #data=self.data, instance = instance, prefix = 'LINES_%s' % pk_value)]
            prefix = 'lines-%s' % pk_value,
        )]