Why does this queryset filter crash silently?

Question

Basically, what i'm trying to do is filter out a company inside of a query, and this code is the result of a lot of debugging. we've reduced it to as simple as possible and can't work out why "for company in Company.objects.filter(pk="E0433CAE-1756-11E1-A97B-7071BC08AB6F"):" fails. anywhere else and it works perfectly :s

Autovivification, is nesting dictionaries automatically. See Best way to automatically nest dictionaries

Code to be run:

def run_code(code, query_dict):
    code = code.replace('\r\n', '\n')
    import jmsdirectory.advert.models
    import jmsdirectory.boughtin.models
    import jmsdirectory.contacts.models
    import jmsdirectory.generic.models
    import jmsdirectory.joinerysoft.models
    import jmsdirectory.scheduler.models
    import jmsdirectory.suppliers.models
    # g = {'__builtins__': False }
    # try:
    print "WOOPS"
    # print g
    print query_dict
    class AutoVivification(dict):
        print "AutoVivification: 1"
        """Implementation of perl's autovivification feature."""
        def __getitem__(self, item):
            try:
                return dict.__getitem__(self, item)
            except KeyError:
                value = self[item] = type(self)()
                return value
    print "AutoVivification: 69"
    models = AutoVivification()
    print "AutoVivification: 71"

    for company in Company.objects.filter(pk="E0433CAE-1756-11E1-A97B-7071BC08AB6F"):
        print "AutoVivification: 74"
        for address in Address.objects.filter(company=query_dict['company']).exclude(allow_download=0):
            print "AutoVivification: 76"
            for contact in Contact.objects.filter(address=address.uuid).exclude(allow_download=0):
                print "AutoVivification: 78"
                for supplier in Supplier.objects.filter(company=query_dict['company']):
                    print "AutoVivification: 80"
                    for supplies in Supplies.objects.filter(supplier=supplier.uuid):
                        print "AutoVivification: 82"
                        models[company][address][contact][supplier] = supplies
                        print "AutoVivification: 84"
                        print models
    print "Rage!"
    result = models
    # exec code in locals()
    print "MEH!"
    # except:
        # print "Error in query set"
        # print traceback.print_exc()
        # return False
    return result

output:

Django version 1.2.7, using settings 'jmsdirectory.settings'
Development server is running at http://127.0.0.1:8002/
Quit the server with CTRL-BREAK.
[21/May/2012 12:22:18] "POST /transfer/server/ HTTP/1.0" 200 162
[21/May/2012 12:22:18] "POST /transfer/server/ HTTP/1.0" 200 531
[21/May/2012 12:22:18] "POST /transfer/server/ HTTP/1.0" 200 162
1
2
3
4
5
6
7
8
9
WOOPS
{'company': u'E0433CAE-1756-11E1-A97B-7071BC08AB6F', 'contact': u'E047B1A8-1756-11E1-A97B-7071BC08AB6F', 'address': u'E0452488-1756-11E1-A97B-7071BC08AB6F'}
AutoVivification: 1
AutoVivification: 69
AutoVivification: 71
[21/May/2012 12:22:18] "POST /transfer/server/ HTTP/1.0" 200 129

model for company:

class Company(models.Model):
    uuid = UUIDField(primary_key=True)
    name =  models.CharField(null=True, blank=True,max_length=255,verbose_name=_('Company Name'))
    internal_name = models.CharField(null=True, blank=True,max_length=255,verbose_name=_('Internal Name'))
    reference = models.CharField(null=True, blank=True,max_length=255,verbose_name=_('Reference'))
    company_status = models.ForeignKey(CompanyStatus, null=True, db_column='company_status_uuid',verbose_name=(_('Company Status')), serialize=False)
    vat_number = models.CharField(null=True, blank=True,max_length=255,verbose_name=(_('Vat Number')))
    registration_number = models.CharField(null=True, blank=True,max_length=255,verbose_name=(_('Company Number')))
    discount = models.FloatField(null=True, blank=True)
    notes = models.TextField(null=True, blank=True,max_length=255)
    jms_code = models.TextField(null=True, blank=True,max_length=255)
    logo = models.TextField(null=True, blank=True,max_length=255)
    date_created = models.DateTimeField(null=True, blank=True, auto_now_add=True,verbose_name=_('Date Time'), serialize=False)
    date_modified = models.DateTimeField(null=True, blank=True, auto_now=True,verbose_name=_('Date Time Updated'), serialize=False)
    hidden = models.NullBooleanField(null=True, blank=True,default=0, serialize=False)
    user = UserField(null=True, blank=True, serialize=False)
    allow_download = models.NullBooleanField()
    is_modified = ModifiedField(serialize=False)
    companyid = models.IntegerField(null=True, blank=True, serialize=False)
    seqno = models.IntegerField(null=True, blank=True, serialize=False)

    def __unicode__(self):
        return self.name

    def advertiser_uuid(self):
        uuid = self.uuid
        today = date.today()

        from jmsdirectory.advert.models import Advertiser
        a=Advertiser.objects.filter(company=self,start_date__lte=today,end_date__gte=today)
        if len(a) > 0:
            return a[0].uuid
        else:
            return ''

    def advertiser(self):
        if self.advertiser_uuid() != '':
            return True
        else:
            return False

    def get_sequence (self) :

        if self.seqno is None :
            from django.db.models import Max
            max = Company.objects.all().aggregate(Max('seqno'))['seqno__max']
            if max is None :
                   self.companyid = 1
            else : self.companyid = max + 1
            self.seqno = 1
        else :
            self.seqno += 1

        self.save()
        return self.companyid, self.seqno

    class Meta:
        verbose_name_plural = ('Companies')
        ordering = ('name',)
        db_table = 'company'

Fixed:

def run_code(code, query_dict):
    code = code.replace('\r\n', '\n')

    class AutoVivification(dict):
        """Implementation of perl's autovivification feature."""
        def __getitem__(self, item):
            try:
                return dict.__getitem__(self, item)
            except KeyError:
                value = self[item] = type(self)()
                return value

    models = AutoVivification()


    companies = Company.objects.filter(pk="E0433CAE-1756-11E1-A97B-7071BC08AB6F")
    addresses = Address.objects.filter(company=query_dict['company']).exclude(allow_download=0)
    suppliers = Supplier.objects.filter(company=query_dict['company'])
    for company in companies:
        for address in addresses:
            for contact in Contact.objects.filter(address=address.uuid).exclude(allow_download=0):
                for supplier in suppliers:
                    for supply in Supplies.objects.filter(supplier=supplier.uuid):
                        models[company][address][contact][supplier] = supply
    result = models

    return result

Can you reduce the example further and still reproduce the problem? — Daniel Watkins, May 21 '12 at 11:44
Also note that when you do get this working, it's going to be **extremely** inefficient. — Daniel Roseman, May 21 '12 at 11:59
Is there a reason you aren't using [`defaultdict`](http://docs.python.org/library/collections.html#collections.defaultdict)? — Burhan Khalid, May 21 '12 at 12:24
defaultdict doesn't allow me to create nested dictionaries as easily — Jharwood, May 21 '12 at 12:48
The company exists for that PK, i made sure, hence being hardwired — Jharwood, May 21 '12 at 12:49

Burhan Khalid · Accepted Answer · 2012-05-21T12:46:56.260

2

The short answer is - it is not crashing, it is simply not returning anything so you are iterating over empty sets, hence nothing gets printed.

The long answer:

I'll try to help you out a bit, but my first suggestion would be to reduce your problem to its basics. For example, use defaultdict instead of your autovivi class.

You also have some other issues that I can see, for example:

for address in Address.objects.filter(company=query_dict['company'])

You are trying to filter on a PK against a string, I think you mean:

for address in Address.objects.filter(company__uuid=query_dict['company'])

as judging by your other models, I suspect your initial query should be:

for company in Company.objects.filter(uuid="E0433CAE-1756-11E1-A97B-7071BC08AB6F")

Similarly, you should fix your other queries:

contact in Contact.objects.filter(address__uuid=address.uuid) OR even better contact in Contact.objects.filter(address=address) if you have the appropriate FK relationships.

However, even if you get this going, as Daniel hinted, this is highly inefficient. You should take advantage of filter chaining and foreign key lookups.

edited May 21 '12 at 12:46

answered May 21 '12 at 12:38

Burhan Khalid

169,990
18
245
284

Oh, but it is crashing, either way it'd print at least something. even if i didn't expect it to print it, every single line had print statements surrounding it, so i can trace execution path in console. Another thing, this is/was already using many foreign keys lookups. i have hardwired in the uuid in attempt to narrow down where the fault is. – Jharwood May 21 '12 at 12:45
If it would have crashed, you have gotten an exception; the reason its not printing anything in your loops is because you are iterating over an empty set. It is equivalent to `for i in []: print i` this won't print anything, but its perfectly valid Python. – Burhan Khalid May 21 '12 at 12:48
then why isn't it printing the line after the for loop? `print "Rage!"` – Jharwood May 21 '12 at 12:50
pk is also short for primary key, so should work correctly, as it does everywhere else :) – Jharwood May 21 '12 at 12:53

Why does this queryset filter crash silently?

1 Answers1