Django import-export new values in foriegn key model

Question

I'm using the django import-export library to data. It works well except I cannot get it to import objects which don't already exist in the foreign key.

If the objects (values in the csv) exist in the foreign key model- then it imports fine.

But if the objects/values don't exist in the foreign key model- it says "matching query does not exist" and will not import the data.

How can I tell it to add new object to the foreign key model if they don't exist in the foreign key?

Models.py snippet

class Store(models.Model):

    store_name = models.CharField(max_length=30)
    def __unicode__(self):
        return self.store_name
    #etc

class Product(models.Model):

    Store = models.ForeignKey(Store)
    Category = models.ForeignKey(Category)
    first_name = models.CharField(max_length=30)
    second_name = models.CharField(max_length=30)
...

Admin.py snippet

admin.site.register(Category)
admin.site.register(Store)

class ProductResource(resources.ModelResource):

     store_name = fields.Field(column_name='store_name', attribute='Store',
                       widget=ForeignKeyWidget(Store, 'store_name'))

    def __unicode__(self):
        return self.store_name.name

    class Meta:
        model = Product
        fields = ('id', 'first_name', 'second_name','store_name')
        export_order = ('id', 'second_name', 'first_name')
        skip_unchanged = False
        report_skipped = False
        widgets = {
                'published': {'format': '%d.%m.%Y'},
                }


class ProductAdmin(ImportExportModelAdmin):
    resource_class = ProductResource
    list_display = ('first_name', 'second_name')

admin.site.register(Product, ProductAdmin)

andilabs · Answer 1 · 2015-09-03T08:52:16.160

2

In ForeignKeyWidget you have method

  def clean(self, value):
        val = super(ForeignKeyWidget, self).clean(value)
        return self.model.objects.get(**{self.field: val}) if val else None

you could try to override it to do something like get_or_create...

it should look something like this...

from import_export.widgets import ForeignKeyWidget
class MyCustomizationToFKWidget(ForeignKeyWidget):

      def clean(self, value):
            val = super(ForeignKeyWidget, self).clean(value)
            HERE SOME LOGIC OVERRIDEN

edited Sep 03 '15 at 08:52

answered Sep 03 '15 at 08:08

andilabs

22,159
14
114
151

Thanks- but where abouts should I put it? This did not work- `store_name = fields.Field(column_name='store_name', attribute='Store', widget=ForeignKeyWidget(Store, 'store_name')) def clean(self, value): val = super(ForeignKeyWidget, self).clean(value) return self.model.objects.get(**{self.field: val}) if val else None` – Ycon Sep 03 '15 at 08:26
1

@Ycon I updated my answer, but you have to put in `HERE SOME LOGIC` your custom `get_or_create` https://docs.djangoproject.com/en/1.8/ref/models/querysets/#get-or-create – andilabs Sep 03 '15 at 08:54
Im not familiar with get_or_create at all. I read the docs it's still unclear. Can you give me an example for my models? I dont understand what logic is. – Ycon Sep 03 '15 at 11:04

score 2 · Answer 2 · answered Jul 03 '17 at 22:57

Provided the ForeignKey of your model can be null :

MyForeignKeyName = Models.ForeignKey(<modelclass>,blank=True, null=True)

You can add a before_import_row() method in your Resource class :

def before_import_row(self,row) :
    fieldname = 'MyForeignKeyName'
    if not( <modelclass>.objects.filter(pk=row[fieldname]).exists() ) :
        # print(row['id'],row[fieldname],'unknown key !') # console log
        row[fieldname ] = None # or something else.. *

*.. compliant with your foreignkey configuration [ python None = SQL null ]

I'm a bit new to Django so maybe it's not the better way, but this solves my problem.

There's also something about db_constraint=False that can be added to ForeignKey's arguments (some info @stackoverflow and @django) but well, I did not find my way with it.

score 2 · Answer 3 · answered Jun 16 '21 at 15:20

I found the best way to do this is with custom widgets. I found a thread on the django-import-export GitHub project here that contained a lot of useful information. From that I pieced together these two custom widgets that support creation of new objects. These should both work as-is by adding them to your admin.py module:

from import_export.widgets import ManyToManyWidget
from django.db.models import QuerySet


class ForeignKeyWidgetWithCreation(ForeignKeyWidget):
    """
    Taken from a GitHub post.
    https://github.com/django-import-export/django-import-export/issues/318#issuecomment-139989178
    """

    def __init__(self, model, field="pk", create=False, **kwargs):
        self.model = model
        self.field = field
        self.create = create
        super(ForeignKeyWidgetWithCreation, self).__init__(model, field=field, **kwargs)

    def clean(self, value, **kwargs):
        if not value:
            return None

        if self.create:
            self.model.objects.get_or_create(**{self.field: value})

        val = super(ForeignKeyWidgetWithCreation, self).clean(value, **kwargs)

        return self.model.objects.get(**{self.field: val}) if val else None


class ManyToManyWidgetWithCreation(ManyToManyWidget):
    """
    Taken from a GitHub post.
    https://github.com/django-import-export/django-import-export/issues/318#issuecomment-139989178
    """

    def __init__(self, model, field="pk", create=False, **kwargs):
        self.model = model
        self.field = field
        self.create = create
        super(ManyToManyWidgetWithCreation, self).__init__(model, field=field, **kwargs)

    def clean(self, value, **kwargs):

        # If no value was passed then we don't have anything to clean.

        if not value:
            return self.model.objects.none()

        # Call the super method. This will return a QuerySet containing any pre-existing objects.
        # Any missing objects will be

        cleaned_value: QuerySet = super(ManyToManyWidgetWithCreation, self).clean(
            value, **kwargs
        )

        # Value will be a string that is separated by `self.separator`.
        # Each entry in the list will be a reference to an object. If the object exists it will
        # appear in the cleaned_value results. If the number of objects in the cleaned_value
        # results matches the number of objects in the delimited list then all objects already
        # exist and we can just return those results.

        object_list = value.split(self.separator)

        if len(cleaned_value.all()) == len(object_list):
            return cleaned_value

        # If we are creating new objects then loop over each object in the list and
        # use get_or_create to, um, get or create the object.

        if self.create:
            for object_value in object_list:
                _instance, _new = self.model.objects.get_or_create(
                    **{self.field: object_value}
                )

        # Use `filter` to re-locate all the objects in the list.

        model_objects = self.model.objects.filter(**{f"{self.field}__in": object_list})

        return model_objects

You can use these in your import-export model resource class like this:

class SomeModelResource(Resource):
    my_fk_field = fields.Field(
        attribute='my_fk_field',
        widget=ForeignKeyWidgetWithCreation(
            model=models.OtherModel,
            field='name',
            create=True))

    my_m2m_field = fields.Field(
        attribute="my_m2m_field",
        widget=ManyToManyWidgetWithCreation(
            model=models.OtherM2MModel,
            field="name",
            separator="|",
            create=True,
        ),
        default="",
    )

score 1 · Answer 4 · answered Mar 25 '20 at 13:52

You can create missing foreign key entries in the before_import_row() method of your Resource class:

class ProductResource(resources.ModelResource):
    ...
    def before_import_row(self, row, **kwargs):
        Store.objects.get_or_create(
            store_name=row.get('store_name')
        )

Django import-export new values in foriegn key model

4 Answers4

Linked