0

I'm using Django-filter on my django-tables2 table, I have a custom filter that uses one search box to search all fields. SiteFilterEx, it gets all the field names and creates or queries to search each field name.

my model has the following:

class Site(models.Model):
    location = models.CharField(max_length=50)
    ref = models.CharField(max_length=255, blank=True, null=True)
    bgp_as = models.CharField(max_length=6, verbose_name="BGP AS Number")
    opening_date = models.DateField(verbose_name="Site opening date", blank=True, null=True)
    tel = models.CharField(max_length=20, blank=True, null=True)
    address = models.CharField(max_length=255, blank=True, null=True)
    town = models.CharField(max_length=255, blank=True, null=True)
    postcode = models.CharField(max_length=10, blank=True, null=True)

class DeviceCircuitSubnets(models.Model):
    device = models.ForeignKey(Device, on_delete=models.CASCADE)
    circuit = models.ForeignKey(Circuit, on_delete=models.CASCADE, blank=True, null=True)
    subnet = models.ForeignKey(Subnet, on_delete=models.CASCADE)
    active_link = models.BooleanField(default=False, verbose_name="Active Link?")
    active_link_timestamp = models.DateTimeField(auto_now=True, blank=True, null=True)

However id like to also search against the annotated field "active_circuit" (which can be seen in the QuerySet) but when I add this to the list of fields, I get errors stating it doesn't exist. I need to somehow pass the annotated field through to the filter as a valid field to search for or something else?

filter:

class SiteFilterEx(django_filters.FilterSet):
    ex = django_filters.CharFilter(label='Ex filter', method='filter_ex')
    search_fields = ['location', 'sitesupernet__subnet', 'bgp_as', 'opening_date','town','postcode','active_circuit']

    def filter_ex(self, qs, name, value):
        if value:
            q_parts = value.split()

            # Use a global q_totals
            q_totals = Q()

            # This part will get us all possible segmantiation of the query parts and put it in the possibilities list
            combinatorics = itertools.product([True, False], repeat=len(q_parts) - 1)
            possibilities = []
            for combination in combinatorics:
                i = 0
                one_such_combination = [q_parts[i]]
                for slab in combination:
                    i += 1
                    if not slab: # there is a join
                        one_such_combination[-1] += ' ' + q_parts[i]
                    else:
                        one_such_combination += [q_parts[i]]
                possibilities.append(one_such_combination)

            # Now, for all possiblities we'll append all the Q objects using OR
            for p in possibilities:
                list1=self.search_fields
                list2=p
                perms = [zip(x,list2) for x in itertools.permutations(list1,len(list2))]

                for perm in perms:
                    q_part = Q()
                    for p in perm:
                        q_part = q_part & Q(**{p[0]+'__icontains': p[1]})
                    q_totals = q_totals | q_part

            qs = qs.filter(q_totals)
        return qs    

    class Meta:
        model = Site
        fields = ['ex']     
        form = SiteFilterForm  

view:

class Sites(LoginRequiredMixin, ExportMixin, PagedFilteredTableView):
    model = Site
    table_class = SiteTable
    template_name = "app_settings/table_view.html"
    login_url = '/login/'
    redirect_field_name = 'redirect_to'
    filter_class = SiteFilterEx
    exclude_columns = ("buttons", )
    table_pagination = {'per_page': 12}

    def dispatch(self, *args, **kwargs):
        site_type = get_object_or_404(SiteType, pk=self.kwargs['site_type'])
        site_state = 'Open' if self.kwargs['state'] else 'Closed'
        self.site_type_name = '{} {}s'.format(site_state, site_type.site_type)
        self.site_type_icon = 'fa {}'.format(site_type.icon)
        return super(Sites, self).dispatch(*args, **kwargs)

    def get_queryset(self):
        site_type = self.kwargs['site_type']
        subnet = Subquery(
            DeviceCircuitSubnets.objects.filter(device__site_id=OuterRef('id'), \
                                        active_link=True, \
                                        circuit__decommissioned=False
                                        ).values('circuit__name')[:1])        
        active_circuit = Subquery(
            DeviceCircuitSubnets.objects.filter(device__site_id=OuterRef('id'), \
                                        active_link=True, \
                                        circuit__decommissioned=False
                                        ).values('circuit__name')[:1])

        return super(Sites, self).get_queryset().filter(
                is_live=self.kwargs['state'], site_type_id=site_type
            ).annotate(
                active_circuit=active_circuit
            ).prefetch_related('sitesupernet_set')


    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)     
        context['page_icon']=self.site_type_icon
        context['page_title']=self.site_type_name 
        return context

Traceback error:

File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  34.             response = get_response(request)

File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  115.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  113.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python3.6/site-packages/django/views/generic/base.py" in view
  71.             return self.dispatch(request, *args, **kwargs)

File "/itapp/itapp/sites/views.py" in dispatch
  127.         return super(Sites, self).dispatch(*args, **kwargs)

File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/mixins.py" in dispatch
  52.         return super().dispatch(request, *args, **kwargs)

File "/usr/local/lib/python3.6/site-packages/django/views/generic/base.py" in dispatch
  97.         return handler(request, *args, **kwargs)

File "/usr/local/lib/python3.6/site-packages/django/views/generic/list.py" in get
  142.         self.object_list = self.get_queryset()

File "/itapp/itapp/sites/views.py" in get_queryset
  142.             return super(Sites, self).get_queryset().filter(

File "/itapp/itapp/itapp/functions.py" in get_queryset
  560.         return self.filter.qs

File "/usr/local/lib/python3.6/site-packages/django_filters/filterset.py" in qs
  237.                 qs = self.filter_queryset(qs)

File "/usr/local/lib/python3.6/site-packages/django_filters/filterset.py" in filter_queryset
  224.             queryset = self.filters[name].filter(queryset, value)

File "/usr/local/lib/python3.6/site-packages/django_filters/filters.py" in __call__
  765.         return self.method(qs, self.f.field_name, value)

File "/itapp/itapp/sites/filters.py" in filter_ex
  53.             qs = qs.filter(q_totals)

File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in filter
  892.         return self._filter_or_exclude(False, *args, **kwargs)

File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in _filter_or_exclude
  910.             clone.query.add_q(Q(*args, **kwargs))

File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py" in add_q
  1290.         clause, _ = self._add_q(q_object, self.used_aliases)

File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py" in _add_q
  1312.                     current_negated, allow_joins, split_subq, simple_col)

File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py" in _add_q
  1318.                     split_subq=split_subq, simple_col=simple_col,

File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py" in build_filter
  1190.         lookups, parts, reffed_expression = self.solve_lookup_type(arg)

File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py" in solve_lookup_type
  1049.         _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())

File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py" in names_to_path
  1420.                                      "Choices are: %s" % (name, ", ".join(available)))

Exception Type: FieldError at /sites/sites/1/2/
Exception Value: Cannot resolve keyword 'active_circuit' into field. Choices are: acc_short, address,  device, downtimeschedule, id, is_live, last_hw_refresh_date, location, no_wallboard, opening_date, postcode, ref, site_type, site_type_id, sitecircuits, sitecontacts, sitefile, sitenotes, sitesupernet, snmpdata, tel, town, watchlist
AlexW
  • 2,843
  • 12
  • 74
  • 156
  • First of all, I am not able to understand what is your code about to do. IMHO please add the query from a business perspective. – JPG Nov 21 '19 at 13:38
  • and, what your mean by *"I cannot currently filter on any annotated fields"*? Have you tried something similar thing? – JPG Nov 21 '19 at 13:39
  • Hi, ive edited and tried to re explain the issue, has this helped? – AlexW Nov 21 '19 at 14:23
  • Can you add the error traceback? Also, Is it possible to get the specified annotated field value from the `filter_ex` method? – JPG Nov 21 '19 at 14:38
  • hi ve added the traceback, what do you mean by get the annotated field from filter_Ex? thank you – AlexW Nov 22 '19 at 12:52
  • I can see a method named **`filter_ex(self, qs, name, value)`** in your **`SiteFilterEx`** class. The parameter **`qs`** will be the queryset from the view. inside that method, put `print(qs[0].active_circuit)` and check the result – JPG Nov 22 '19 at 13:01
  • During annotation of a field "active_circuit" Did you try mentioning what sort of field i.e what is the type of the field. from django.db import models ; models.IntegerField(). – redhatvicky Nov 25 '19 at 10:37
  • Does it work when you do not make a `SubQuery` and instead take `string` from first query and annotate it in second? – Marek Nov 25 '19 at 11:17
  • I get site has no field active_circuit – AlexW Nov 25 '19 at 15:37

1 Answers1

0

Try to Mention a type for the Subquery , the models can be imported from django.db import models

 active_circuit = Subquery(
                DeviceCircuitSubnets.objects.filter(device__site_id=OuterRef('id'), \
                                            active_link=True, \
                                            circuit__decommissioned=False
                                            ).values('circuit__name')[:1],output_field=models.CharField())
redhatvicky
  • 1,912
  • 9
  • 8
  • I don't think it will be `IntegerField`, probably `CharField`. – Marek Nov 25 '19 at 11:16
  • Yeah Its a "circuit__name" updated the IntegerField to CharField . Thank You . – redhatvicky Nov 25 '19 at 12:20
  • same error, Cannot resolve keyword 'active_circuit' into field – AlexW Nov 25 '19 at 15:38
  • is there any chance that the subquery result has many to one relation ship ? , There was also one more suggestion that popped up suddenly , can you define a object model "circuit__name " and try this out ? Lets see if this gets resolved we would able to drill down the issue and lets try to fix that in a more speculated approach !!! – redhatvicky Nov 25 '19 at 18:18
  • ive added the DeviceCircuitSubnets model to the question incase that helps? – AlexW Nov 27 '19 at 09:30
  • Yeah , can you help us with the circuit model also as circuit is a foreign key in the DeviceCircuitSubnets and you are trying to access "circuit__name" – redhatvicky Nov 27 '19 at 12:56
  • Moreover , if you are using "related_name" you can access directly or else to access foreign you should use _set for example : in your case circuit = models.ForeignKey(Circuit, on_delete=models.CASCADE, blank=True, null=True,related_name="circut_name") and then you can use "circut_name" in subquery else the default would be "circut_set" – redhatvicky Nov 27 '19 at 13:02