create fixtures with custom manager methods, json dumps and ways to avoid type error :xxx is not json serializable

Question

I'm trying to create a test fixture using custom manager methods as my app uses a subset of dbtables and fewer records. so i dropped the idea of using initial_data. In manager I'm doing something like this. in Managers.py:

sitedict = Site.objects.filter(pk=1234).values()[0]
custdict = Customer.objects.filter(custid=123456).values()[0]
customer = {"pk":123456,"model":"myapp.customer","fields":custdict}
site = {"pk":0001,"model":"myapp.site","fields":sitedict}
csvfile = open('shoppingcart/bsofttestdata.csv','wb')
csv_writer = csv.writer(csvfile) 
csv_writer.writerow([customer,site])

then i did modify my csv file to replace single quotes with double, etc. Then i did save that file as json.Sorry if its too dumb way but this is the first time I'm creating testdata,I'd love to learn better way.Sample data of the file is like this in : myapp/fixtures/testdata.json

[{"pk": 123456, "model": "myapp.customer", "fields": {"city": "abc", "maritalstatus": None, "zipcode": "12345", "lname": "fdfdf", "state": "AZ", "agentid": 1111, "fname": "sdadsad", "email": "abcd@xxx.com", "phone": "0000000000", "custid":123456,"datecreate": datetime.datetime(2011, 3, 29, 11, 40, 18, 157612)}},{"pk":0001, "model": "myapp.site", "fields": {"url": "http://google.com", "websitecode": "", "notify": True, "fee": 210.0, "id":0001}}]

I used this to run my tests but i got the following error:

EProblem installing fixture '/var/lib/django/myproject/myapp/fixtures/testdata.json':    
    Traceback (most recent call last):
      File "/usr/lib/pymodules/python2.6/django/core/management/commands/loaddata.py", line 150, in handle
        for obj in objects:
      File "/usr/lib/pymodules/python2.6/django/core/serializers/json.py", line 41, in Deserializer
        for obj in PythonDeserializer(simplejson.load(stream)):
      File "/usr/lib/pymodules/python2.6/simplejson/__init__.py", line 267, in load
        parse_constant=parse_constant, **kw)
      File "/usr/lib/pymodules/python2.6/simplejson/__init__.py", line 307, in loads
        return _default_decoder.decode(s)
      File "/usr/lib/pymodules/python2.6/simplejson/decoder.py", line 335, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib/pymodules/python2.6/simplejson/decoder.py", line 353, in raw_decode
        raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded

score 1 · Accepted Answer · edited May 23 '17 at 12:11

in stead of using raw find replace its better to use something as shown here and when we have some datatypes that JSON doesn't support.this would be helpful to get rid of TypeError: xxxxxxx is not JSON serializable or specifically stackover post for Datetime problem will be helpful.

EDIT: instead of writing to csv then manually modifying it,I did the following:

with open('myapp/fixtures/customer_testdata.json',mode = 'w') as f:
    json.dump(customer,f,indent=2)

here is small code I used to get out of the TypeError:xxxx not json blah blah problem

for key in cust.keys():
    value = cust[key]
    if isinstance(cust[key],datetime.datetime):
        temp = cust[key].timetuple() # this converts datetime.datetime to time.struct_time
        cust.update({key:{'__class__':'time.asctime','__value__':time.asctime(temp)}})
return cust

if we convert datetime.datetime to any other type, then we have to chang the class accordingly. E.g timestamp --> float here is fantastic reference for datetime conversions

Hope this is helpful.

create fixtures with custom manager methods, json dumps and ways to avoid type error :xxx is not json serializable

1 Answers1